-- Logs begin at Wed 2022-07-06 16:04:59 UTC, end at Wed 2022-07-06 18:05:09 UTC. -- Jul 06 16:53:18.540608 dv-cinder-806605-6-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 06 16:53:18.549625 dv-cinder-806605-6-iscsi devstack@keystone.service[80455]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 06 16:53:18.550999 dv-cinder-806605-6-iscsi devstack@keystone.service[80455]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jul 6 16:53:18 2022] *** Jul 06 16:53:18.550999 dv-cinder-806605-6-iscsi devstack@keystone.service[80455]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 06 16:53:18.550999 dv-cinder-806605-6-iscsi devstack@keystone.service[80455]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 06 16:53:18.550999 dv-cinder-806605-6-iscsi devstack@keystone.service[80455]: nodename: dv-cinder-806605-6-iscsi Jul 06 16:53:18.550999 dv-cinder-806605-6-iscsi devstack@keystone.service[80455]: machine: x86_64 Jul 06 16:53:18.550999 dv-cinder-806605-6-iscsi devstack@keystone.service[80455]: clock source: unix Jul 06 16:53:18.550999 dv-cinder-806605-6-iscsi devstack@keystone.service[80455]: pcre jit disabled Jul 06 16:53:18.550999 dv-cinder-806605-6-iscsi devstack@keystone.service[80455]: detected number of CPU cores: 4 Jul 06 16:53:18.550999 dv-cinder-806605-6-iscsi devstack@keystone.service[80455]: current working directory: / Jul 06 16:53:18.550999 dv-cinder-806605-6-iscsi devstack@keystone.service[80455]: detected binary path: /usr/bin/uwsgi-core Jul 06 16:53:18.550999 dv-cinder-806605-6-iscsi devstack@keystone.service[80455]: your processes number limit is 47680 Jul 06 16:53:18.550999 dv-cinder-806605-6-iscsi devstack@keystone.service[80455]: your memory page size is 4096 bytes Jul 06 16:53:18.550999 dv-cinder-806605-6-iscsi devstack@keystone.service[80455]: detected max file descriptor number: 2048 Jul 06 16:53:18.550999 dv-cinder-806605-6-iscsi devstack@keystone.service[80455]: lock engine: pthread robust mutexes Jul 06 16:53:18.551719 dv-cinder-806605-6-iscsi devstack@keystone.service[80455]: thunder lock: enabled Jul 06 16:53:18.551719 dv-cinder-806605-6-iscsi devstack@keystone.service[80455]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 06 16:53:18.551719 dv-cinder-806605-6-iscsi devstack@keystone.service[80455]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 06 16:53:18.595392 dv-cinder-806605-6-iscsi devstack@keystone.service[80455]: Python main interpreter initialized at 0x55b881e179c0 Jul 06 16:53:18.595392 dv-cinder-806605-6-iscsi devstack@keystone.service[80455]: python threads support enabled Jul 06 16:53:18.595392 dv-cinder-806605-6-iscsi devstack@keystone.service[80455]: your server socket listen backlog is limited to 100 connections Jul 06 16:53:18.595392 dv-cinder-806605-6-iscsi devstack@keystone.service[80455]: your mercy for graceful operations on workers is 90 seconds Jul 06 16:53:18.595797 dv-cinder-806605-6-iscsi devstack@keystone.service[80455]: mapped 403077 bytes (393 KB) for 2 cores Jul 06 16:53:18.595912 dv-cinder-806605-6-iscsi devstack@keystone.service[80455]: *** Operational MODE: preforking *** Jul 06 16:53:18.596108 dv-cinder-806605-6-iscsi devstack@keystone.service[80455]: *** uWSGI is running in multiple interpreter mode *** Jul 06 16:53:18.596108 dv-cinder-806605-6-iscsi devstack@keystone.service[80455]: spawned uWSGI master process (pid: 80455) Jul 06 16:53:18.596313 dv-cinder-806605-6-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 06 16:53:18.597964 dv-cinder-806605-6-iscsi devstack@keystone.service[80455]: spawned uWSGI worker 1 (pid: 80469, cores: 1) Jul 06 16:53:18.597964 dv-cinder-806605-6-iscsi devstack@keystone.service[80455]: spawned uWSGI worker 2 (pid: 80470, cores: 1) Jul 06 16:53:18.597964 dv-cinder-806605-6-iscsi devstack@keystone.service[80455]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 06 16:53:24.029733 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 06 16:53:24.029733 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: from cryptography.utils import int_from_bytes Jul 06 16:53:24.030485 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 06 16:53:24.030485 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: from cryptography.utils import int_from_bytes Jul 06 16:53:24.030597 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 06 16:53:24.030597 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: from cryptography.utils import int_from_bytes Jul 06 16:53:24.030597 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 06 16:53:24.030597 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: from cryptography.utils import int_from_bytes Jul 06 16:53:24.078993 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 06 16:53:24.079204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 06 16:53:24.079394 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 06 16:53:24.079492 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] command line args: [] {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 06 16:53:24.079615 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 06 16:53:24.079697 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 06 16:53:24.079827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] command line args: [] {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 06 16:53:24.079909 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ================================================================================ {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 06 16:53:24.080099 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 06 16:53:24.080179 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] admin_token = **** {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.080281 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ================================================================================ {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 06 16:53:24.080368 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] config_dir = [] {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.080513 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.080599 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] admin_token = **** {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.080678 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] config_source = [] {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.080771 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] config_dir = [] {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.080854 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] debug = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.080961 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.081047 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.081284 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] config_source = [] {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.081284 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] debug = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.081412 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.081545 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.081660 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.081729 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.081788 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.081911 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.082036 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.082130 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.082268 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] list_limit = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.082388 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.082470 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] log_config_append = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.082606 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] list_limit = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.082691 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.082761 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] log_dir = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.082826 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] log_config_append = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.082941 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] log_file = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.083017 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.083121 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] log_dir = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.083199 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.083271 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] log_file = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.083397 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.083522 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.083651 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.083750 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.083923 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.084023 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.084098 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.084222 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.084307 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.084403 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.084494 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.084567 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.084717 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.084800 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.084947 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.085032 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.085162 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.085245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.085394 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.085515 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.085622 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.085699 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.085874 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.085957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.086114 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.086242 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.086370 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.086454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.086652 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.086852 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.40/identity {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.087034 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] publish_errors = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.087159 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.40/identity {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.087269 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.087353 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] publish_errors = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.087502 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.087627 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.087712 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.087817 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.087934 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.088009 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.088142 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.088268 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.088380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.088458 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] standard_threads = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.088608 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.088685 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] standard_threads = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.088838 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.088922 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.088994 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.089069 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.089172 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] use_journal = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.089255 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.089339 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] use_json = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.089412 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] use_journal = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.089488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] use_stderr = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.089559 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] use_json = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.089664 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] use_syslog = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.089742 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] use_stderr = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.089889 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] use_syslog = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.089963 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.090103 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 06 16:53:24.090188 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.090360 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.090444 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.090602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.090680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.090874 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.090950 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.091123 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.091206 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.091359 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.091444 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.091606 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.091690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.091867 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.091954 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.092107 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.092197 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.092352 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.092429 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.092599 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.092676 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.092829 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.092908 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.093071 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.093147 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.093303 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.093380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.093545 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.093624 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.093789 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.093867 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.094025 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.094113 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.094263 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.094341 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.094518 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.094594 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.094762 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.094848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.094996 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.095081 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.095232 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.095310 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.095489 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.095643 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.095742 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.095874 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.096012 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.096138 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.096243 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.096394 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.096473 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.096636 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.096720 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.096871 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.096955 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.097105 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.097190 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.097344 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.097423 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.097587 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.097664 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.097816 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.097894 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.098055 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.098132 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.098290 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.098373 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.098523 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.098601 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.098769 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.098877 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.099000 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.099112 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.099220 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.099318 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.099453 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.099560 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.099691 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.099833 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.099923 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.100042 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.100141 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.100220 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.100385 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.100462 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.100620 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.100703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.100854 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.100937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.101094 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.101194 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.101379 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.101469 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.101617 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.101701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.101856 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.101944 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.102089 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.102228 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.102350 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.102424 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.102554 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.102632 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.102793 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.102891 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.103024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.103101 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.103258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.103335 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.103494 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.103582 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.103752 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.103834 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.103981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.104059 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.104218 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.104305 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.104451 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.104531 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.104684 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.104762 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.104915 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.104991 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.105164 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.105242 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.105397 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.105474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.105627 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.105714 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.105868 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.105947 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.106100 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.106177 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.106339 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.106916 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.107098 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.107178 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.107345 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.107422 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.107591 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.107776 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.107854 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.107926 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.108083 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.108162 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.108319 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.108409 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.108557 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.108634 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.108790 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.108867 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.109028 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.109123 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.109272 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.109359 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.109508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.109586 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.109752 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.109836 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.109986 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.110072 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.110223 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.110302 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.110459 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.110538 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.110697 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.110775 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.110937 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.111018 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.111177 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.111262 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.111413 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.111496 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.111654 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.111757 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.111915 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.111994 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.112151 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.112232 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.112386 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.112465 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.112616 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.112695 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.112845 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.112924 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.113074 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.113153 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.113307 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.113384 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.113537 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.113617 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.113771 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.113849 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.114005 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.114090 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.114247 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.114326 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.114479 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.114560 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.114727 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.114804 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.114965 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.115045 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.115207 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.115284 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.115436 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] auth.external = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.115515 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.115665 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.115761 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.115943 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] auth.external = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.116021 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.116177 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.116294 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.116423 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] auth.password = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.116502 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.116653 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] auth.token = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.116733 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.116890 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.116969 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] auth.password = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.117124 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.117203 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] auth.token = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.117365 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.117445 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.117592 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.117670 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.117837 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.117915 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.118069 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.118148 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.118296 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.118374 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.118529 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] credential.caching = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.118607 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.118773 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.118853 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.119007 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] credential.caching = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.119085 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.119250 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.119342 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.119486 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.119567 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.119797 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.119883 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.119984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.120063 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.120226 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.120306 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.120466 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.120545 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.120708 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.120786 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.120987 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.121075 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.121194 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.121280 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.121450 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.121607 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.121720 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.121800 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.121963 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.122042 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.122784 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.122887 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.123023 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] federation.caching = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.123104 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.123262 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.123342 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.123508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.123601 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] federation.caching = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.123774 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.123858 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.124010 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.124089 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.124257 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.124350 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.124493 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.124574 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.124741 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.124822 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.124976 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.125067 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.125226 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.125306 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.125462 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.125541 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.125705 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.125793 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.125940 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] identity.caching = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.126021 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.126186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.126273 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.126422 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] identity.caching = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.126509 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.126659 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.126737 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.126894 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.126974 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.127138 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.127228 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.127366 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.127445 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.127602 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.127682 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.127864 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.127956 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.128106 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.128186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.128335 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.128459 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.128559 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.128671 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.128782 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.128863 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.129023 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.129103 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.129258 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.129345 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.129500 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.129579 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.129745 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.129824 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.129986 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.130074 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.130240 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.130320 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.130474 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.130552 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.130709 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.130791 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.130939 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.131019 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.131172 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.131259 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.131403 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.131484 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.131639 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.131717 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.131898 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.131979 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.147577 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.147702 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.147702 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.147969 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.148047 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.148120 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.148120 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.148266 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.148266 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.148395 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.148395 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.148523 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.148604 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.148678 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.148750 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.148822 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.148893 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.148964 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.149045 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.149119 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.149191 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.149269 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.149342 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.149414 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.149494 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.149575 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.149649 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.149721 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.149793 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.149864 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.149943 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.150023 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.150096 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.150168 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.150240 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.150312 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.150391 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.150465 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.150537 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.150609 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.150681 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.150752 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.150831 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.150906 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.150984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.151057 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.151129 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.151200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.151280 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.151354 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.151432 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.user = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.151505 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.151577 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.151649 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.151747 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.151826 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.user = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.151893 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.151964 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.152035 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.152121 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.152205 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.152279 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.152352 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.152424 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.152495 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.152567 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.152647 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.152722 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.152794 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.152865 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.152937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.153009 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.153089 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.153163 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.153235 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.153306 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.153377 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.153448 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.153528 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.153607 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.153680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.153752 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.153824 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.153895 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.153969 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.154042 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.154114 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.154185 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.154263 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.154336 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.154416 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.154491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.154563 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.154633 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.154705 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.154776 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.154856 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.154930 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.155002 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.155074 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.155146 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.155217 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.155286 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.155385 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.155458 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.155519 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.155577 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.155647 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.155818 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.155898 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.155970 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.156042 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.156114 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.156186 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.156266 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.156341 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.156413 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.156484 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.156561 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.156643 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.156717 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.156717 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.156844 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.156916 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] resource.caching = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.156987 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.156987 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.157127 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.157200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.157271 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.157343 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] resource.caching = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.157414 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.157414 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.157552 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.157552 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.157680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.157680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.157807 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.157886 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.157960 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.158031 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.158102 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.158173 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.158244 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.158323 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] role.caching = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.158397 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.158468 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] role.driver = sql {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.158539 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.158610 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.158681 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.158760 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.158834 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] role.caching = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.158906 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.158984 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] role.driver = sql {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.159057 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.159129 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.159209 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.159283 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.159354 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.159425 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.159497 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.159568 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.159647 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.159721 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.159804 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.159876 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.159947 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.160012 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.160096 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.160170 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.160242 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.160314 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.160385 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.160455 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.160535 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.160609 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.160680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.160751 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.160822 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.160893 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.160973 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.161047 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.161119 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.161190 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.161261 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.161341 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.161403 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.161505 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.161567 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.161638 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.161709 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.161788 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.161863 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.161928 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.162000 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.162075 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.162147 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.162232 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.162309 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.162309 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.162426 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.162497 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.162568 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.162648 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.162722 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.162794 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.162859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.162859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.162975 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.162975 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.163114 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.163193 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.163265 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.163337 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] token.caching = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.163407 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.163488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.163562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.163633 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.163704 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.163797 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.163870 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.163954 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.164030 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] token.caching = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.164102 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.164173 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.164244 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.164315 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.164395 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.164468 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.164540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.164611 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.164682 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.164752 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.164832 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.164905 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.164978 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.165048 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.165119 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.165190 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.165270 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.165343 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.165414 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.165485 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.165557 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.165628 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.165708 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.165781 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.165853 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.165924 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.165994 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] database.connection = **** {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.166074 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.166074 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.166221 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.166221 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.166352 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.166446 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.166511 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.166616 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.166708 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] database.connection = **** {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.166792 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.166877 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.166952 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.167024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.167096 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.167167 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.167238 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.167388 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.167465 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.167538 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.167610 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.167682 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.167769 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.167856 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.167931 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.168003 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.168069 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.168140 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.168212 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.168291 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.168365 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.168437 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.168508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.168579 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.168650 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.168730 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.168804 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.168804 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80470) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 06 16:53:24.169415 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.169415 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.169415 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.169415 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.169415 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 06 16:53:24.169415 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80469) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 06 16:53:24.196813 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.196948 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.197032 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.197110 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.197253 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.197347 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.205546 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.205655 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.233775 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.233952 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.234059 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.234169 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.374792 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.375036 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.423963 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.424433 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.424548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.424548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.487606 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.487807 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.487895 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.488040 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.488123 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.488263 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.488346 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.488473 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.488555 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.488687 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.488779 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.488900 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.513108 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.513248 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.513335 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.513431 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.513550 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.513637 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.513775 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.513862 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.513995 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.514090 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.514208 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.514291 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.514419 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.514502 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.525703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.525805 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.538683 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.538790 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.538915 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.539016 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.539140 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.539224 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.539362 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.539445 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.539579 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.539673 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.539822 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.539908 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.540038 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80470) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.540121 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80469) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 06 16:53:24.557039 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.557246 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 06 16:53:24.557675 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.557792 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.557876 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 06 16:53:24.557993 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 06 16:53:24.558208 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.558291 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.558411 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 06 16:53:24.558497 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.558683 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.558776 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.558944 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.559026 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.559203 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.559988 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.570227 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.570500 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.570772 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.570854 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.571049 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.571132 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.571337 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.571620 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.579320 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80470) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 16:53:24.580006 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80469) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 16:53:24.582590 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80470) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 16:53:24.582868 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.583147 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.583275 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80469) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 16:53:24.583420 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.583551 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.583697 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.583847 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.583974 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.584120 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.584234 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.584402 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.584497 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.584663 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.584744 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.584929 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.585010 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.585202 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.585284 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.585471 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.585741 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.586008 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.612568 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.613603 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.614167 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80470) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 16:53:24.614441 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.615220 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80469) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 16:53:24.615499 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.619664 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80470) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 16:53:24.619962 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.620238 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.620776 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80469) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 16:53:24.621058 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.621460 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.627654 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80470) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 16:53:24.627917 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.628760 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80469) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 16:53:24.629042 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.632355 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80470) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 16:53:24.632632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.632906 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.633180 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.633451 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.633661 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80469) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 16:53:24.633753 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.633947 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.634030 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.634218 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.634305 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.634495 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.634768 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.635037 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.635304 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.635571 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.652936 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.653721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.655116 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.655393 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.658256 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80470) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 16:53:24.659973 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80469) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 16:53:24.661879 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.662169 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.663591 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.663912 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.667286 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80470) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 16:53:24.669563 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80469) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 16:53:24.671499 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80470) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 16:53:24.673183 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80469) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 16:53:24.675131 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.675423 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.675710 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.676021 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.676305 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.676578 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.676822 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.677121 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.677408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.677699 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.677994 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.678275 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.695465 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80470) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 16:53:24.695761 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.696029 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.696289 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.697256 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80469) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 16:53:24.697540 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.697806 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.698067 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.704181 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.705870 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.706033 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.706144 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.707749 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.708033 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.709326 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80470) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 16:53:24.709598 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.709877 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.710155 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.710480 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.711194 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80469) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 16:53:24.711476 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.711774 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.712059 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.712341 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.724173 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80470) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 16:53:24.724452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.724730 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.725008 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.725282 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.725569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.725847 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.726095 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80469) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 16:53:24.726374 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.726653 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.726932 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.727211 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.728840 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.729135 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.745410 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80470) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 16:53:24.747602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80469) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 16:53:24.748694 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80470) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 16:53:24.750892 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80469) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 16:53:24.751988 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.753588 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.754235 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.755334 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80470) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 16:53:24.755610 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.755930 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.756015 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.757624 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80469) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 16:53:24.757904 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.758192 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.763252 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.764877 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80470) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 16:53:24.765613 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.767223 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80469) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 16:53:24.768094 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.768381 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.768662 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.768937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.770470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.770759 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.771407 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.771683 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.778870 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80470) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 16:53:24.779150 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.779427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.781429 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80469) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 16:53:24.781715 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.781992 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.787238 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80470) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 16:53:24.787514 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.787814 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.788088 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.788355 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.788638 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.788906 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.789172 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.789529 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.789796 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.789876 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80469) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 06 16:53:24.790069 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.790159 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.790336 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.790420 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.790598 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.790678 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.790860 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80470) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.790943 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.791257 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.791535 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.791838 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.792112 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.792390 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.792678 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.792948 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.793213 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.793480 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80469) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 06 16:53:24.828779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55b881e179c0 pid: 80470 (default app) Jul 06 16:53:24.831686 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55b881e179c0 pid: 80469 (default app) Jul 06 16:53:24.832144 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d59d7-3dca-4022-a12f-09e6ca2d4d96 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:24.832437 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d59d7-3dca-4022-a12f-09e6ca2d4d96 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:24.832728 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d59d7-3dca-4022-a12f-09e6ca2d4d96 None None] PATH_INFO: `/v3` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:24.836965 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1/1] 10.222.0.40 () {50 vars in 791 bytes} [Wed Jul 6 16:53:24 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:53:33.345564 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db345c11-8849-4e29-8ead-424f5b836cd6 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:33.346128 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db345c11-8849-4e29-8ead-424f5b836cd6 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:33.346128 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db345c11-8849-4e29-8ead-424f5b836cd6 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:33.350499 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1/2] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:53:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:53:33.356038 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856b2ff-c331-4788-8d6f-5c183fba0ec6 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:33.359578 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856b2ff-c331-4788-8d6f-5c183fba0ec6 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:33.359578 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2856b2ff-c331-4788-8d6f-5c183fba0ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:33.424261 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: INFO dbcounter [None req-2856b2ff-c331-4788-8d6f-5c183fba0ec6 None None] Registered counter for database keystone Jul 06 16:53:33.444976 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG oslo_db.sqlalchemy.engines [None req-2856b2ff-c331-4788-8d6f-5c183fba0ec6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80470) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 06 16:53:33.451626 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writer thread running {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 06 16:53:33.627575 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG passlib.handlers.bcrypt [None req-2856b2ff-c331-4788-8d6f-5c183fba0ec6 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80470) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 06 16:53:33.627950 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG passlib.handlers.bcrypt [None req-2856b2ff-c331-4788-8d6f-5c183fba0ec6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80470) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 06 16:53:33.682139 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-2856b2ff-c331-4788-8d6f-5c183fba0ec6 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:53:33.683792 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-2856b2ff-c331-4788-8d6f-5c183fba0ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:53:33.804510 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 2/3] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:53:33 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 449 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 06 16:53:34.008460 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db345c11-8849-4e29-8ead-424f5b836cd6 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:34.079234 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: INFO dbcounter [None req-db345c11-8849-4e29-8ead-424f5b836cd6 None None] Registered counter for database keystone Jul 06 16:53:34.098332 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG oslo_db.sqlalchemy.engines [None req-db345c11-8849-4e29-8ead-424f5b836cd6 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80469) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 06 16:53:34.104385 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writer thread running {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 06 16:53:34.221612 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db345c11-8849-4e29-8ead-424f5b836cd6 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:34.225924 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74663b3-ca04-48e3-9ebe-fec883882a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:34.226593 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74663b3-ca04-48e3-9ebe-fec883882a08 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:34.226876 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74663b3-ca04-48e3-9ebe-fec883882a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:34.227152 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74663b3-ca04-48e3-9ebe-fec883882a08 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:34.235968 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74663b3-ca04-48e3-9ebe-fec883882a08 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:34.287820 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c74663b3-ca04-48e3-9ebe-fec883882a08 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 06 16:53:34.288110 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74663b3-ca04-48e3-9ebe-fec883882a08 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:34.298103 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 2/4] 10.222.0.40 () {58 vars in 1324 bytes} [Wed Jul 6 16:53:34 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 16:53:34.304128 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856b2ff-c331-4788-8d6f-5c183fba0ec6 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:34.318409 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2856b2ff-c331-4788-8d6f-5c183fba0ec6 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:34.322778 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43d916d-c2e4-480e-9251-4f5b6c70e855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:34.323915 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d916d-c2e4-480e-9251-4f5b6c70e855 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:34.324208 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d916d-c2e4-480e-9251-4f5b6c70e855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:34.324488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d916d-c2e4-480e-9251-4f5b6c70e855 admin admin] PATH_INFO: `/v3/projects/18f61d57ca574df5935e41a1653e029c` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:34.325973 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43d916d-c2e4-480e-9251-4f5b6c70e855 admin admin] RBAC: Authorizing `identity:get_project(project_id=18f61d57ca574df5935e41a1653e029c)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:34.378750 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d43d916d-c2e4-480e-9251-4f5b6c70e855 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 06 16:53:34.379279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43d916d-c2e4-480e-9251-4f5b6c70e855 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:34.383648 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 3/5] 10.222.0.40 () {56 vars in 1207 bytes} [Wed Jul 6 16:53:34 2022] GET /identity/v3/projects/18f61d57ca574df5935e41a1653e029c => generated 344 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:53:36.360139 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce6062-2e52-44bb-aaff-52ae99aa4104 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:36.360705 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce6062-2e52-44bb-aaff-52ae99aa4104 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:36.360705 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce6062-2e52-44bb-aaff-52ae99aa4104 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:36.364817 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 3/6] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:53:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:53:36.370376 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e101e-452f-4e82-ba17-aa5173daa7d8 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:36.370899 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e101e-452f-4e82-ba17-aa5173daa7d8 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:36.370995 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2e101e-452f-4e82-ba17-aa5173daa7d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:36.390038 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-6f2e101e-452f-4e82-ba17-aa5173daa7d8 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:53:36.391577 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-6f2e101e-452f-4e82-ba17-aa5173daa7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:53:36.399634 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 4/7] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:53:36 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 06 16:53:36.422128 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce6062-2e52-44bb-aaff-52ae99aa4104 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:36.433909 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce6062-2e52-44bb-aaff-52ae99aa4104 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:36.438322 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4174c664-c6b0-475e-839e-d0856d602f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:36.438970 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4174c664-c6b0-475e-839e-d0856d602f87 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:36.439263 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4174c664-c6b0-475e-839e-d0856d602f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:36.439548 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4174c664-c6b0-475e-839e-d0856d602f87 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:36.447300 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4174c664-c6b0-475e-839e-d0856d602f87 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:36.449125 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4174c664-c6b0-475e-839e-d0856d602f87 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:36.458408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 4/8] 10.222.0.40 () {58 vars in 1324 bytes} [Wed Jul 6 16:53:36 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 16:53:36.463467 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e101e-452f-4e82-ba17-aa5173daa7d8 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:36.472946 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2e101e-452f-4e82-ba17-aa5173daa7d8 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:36.477280 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83054244-6988-4d07-8e3b-d7f6131e92c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:36.478138 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83054244-6988-4d07-8e3b-d7f6131e92c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:36.478424 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83054244-6988-4d07-8e3b-d7f6131e92c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:36.478704 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83054244-6988-4d07-8e3b-d7f6131e92c4 admin admin] PATH_INFO: `/v3/users/fb49b9dd86cd415bb41412ca15c4d0ba` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:36.480120 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83054244-6988-4d07-8e3b-d7f6131e92c4 admin admin] RBAC: Authorizing `identity:get_user(user_id=fb49b9dd86cd415bb41412ca15c4d0ba)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:36.481955 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83054244-6988-4d07-8e3b-d7f6131e92c4 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:36.483498 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 5/9] 10.222.0.40 () {56 vars in 1198 bytes} [Wed Jul 6 16:53:36 2022] GET /identity/v3/users/fb49b9dd86cd415bb41412ca15c4d0ba => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:53:38.566160 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10753452-1922-40e2-a14e-8ce565a9a596 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:38.566701 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10753452-1922-40e2-a14e-8ce565a9a596 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:38.566701 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10753452-1922-40e2-a14e-8ce565a9a596 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:38.568192 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 5/10] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:53:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 16:53:38.573386 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928c74a-d138-4309-93dc-58344106d6ac None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:38.573789 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928c74a-d138-4309-93dc-58344106d6ac None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:38.574073 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9928c74a-d138-4309-93dc-58344106d6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:38.590019 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786093e0-9b1d-4660-a5d4-124a57940b4f None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:38.590306 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786093e0-9b1d-4660-a5d4-124a57940b4f None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:38.590445 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-9928c74a-d138-4309-93dc-58344106d6ac None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:53:38.590589 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786093e0-9b1d-4660-a5d4-124a57940b4f None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:38.591635 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-9928c74a-d138-4309-93dc-58344106d6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:53:38.591924 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 6/11] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:53:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 16:53:38.597004 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5412ff61-aa6e-4237-9113-199b82fb3244 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:38.597293 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5412ff61-aa6e-4237-9113-199b82fb3244 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:38.597576 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5412ff61-aa6e-4237-9113-199b82fb3244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:38.637353 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG passlib.handlers.bcrypt [None req-5412ff61-aa6e-4237-9113-199b82fb3244 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80469) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 06 16:53:38.637787 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG passlib.handlers.bcrypt [None req-5412ff61-aa6e-4237-9113-199b82fb3244 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80469) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 06 16:53:38.672231 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 6/12] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:53:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:53:38.683235 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-5412ff61-aa6e-4237-9113-199b82fb3244 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:53:38.685392 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-5412ff61-aa6e-4237-9113-199b82fb3244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:53:38.694828 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9928c74a-d138-4309-93dc-58344106d6ac None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:38.718213 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:38.785728 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 7/13] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:53:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 16:53:38.801682 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9928c74a-d138-4309-93dc-58344106d6ac None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:38.808158 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5412ff61-aa6e-4237-9113-199b82fb3244 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:38.834698 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fc5752-c829-4d94-9971-25ae9f43646b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:38.835429 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc5752-c829-4d94-9971-25ae9f43646b None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:38.835794 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc5752-c829-4d94-9971-25ae9f43646b None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:38.836074 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc5752-c829-4d94-9971-25ae9f43646b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:38.847605 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fc5752-c829-4d94-9971-25ae9f43646b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:53:38.859428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-40fc5752-c829-4d94-9971-25ae9f43646b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:53:38.859428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:53:38.859428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:53:38.859428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:53:38.859428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:53:38.859428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:53:38.859428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:53:38.859428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:53:38.859428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:53:38.859428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:53:38.859428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:53:38.859428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:53:38.859428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:53:38.859428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:53:38.859428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:53:38.859428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:53:38.860521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:53:38.860521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:53:38.860521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:53:38.860521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:53:38.860521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:53:38.860521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:53:38.860521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:53:38.860521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:53:38.860521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:53:38.860521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:53:38.860521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:53:38.860521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:53:38.860521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:53:38.860521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:53:38.860521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:53:38.860521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:53:38.861376 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:53:38.861376 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:53:38.861376 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:53:38.861376 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:53:38.861376 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:53:38.861376 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:53:38.861376 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:53:38.861376 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:53:38.861376 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:53:38.861376 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:53:38.861376 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:53:38.861376 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:53:38.861376 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 7/14] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 6 16:53:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:53:38.866351 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fc5752-c829-4d94-9971-25ae9f43646b None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:38.910416 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5412ff61-aa6e-4237-9113-199b82fb3244 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:38.912701 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:38.947153 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:38.947308 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b164641e-a84d-44fc-9e70-32148f7098a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:38.948206 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164641e-a84d-44fc-9e70-32148f7098a9 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:38.948498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164641e-a84d-44fc-9e70-32148f7098a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:38.948777 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b164641e-a84d-44fc-9e70-32148f7098a9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:38.964257 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fc5752-c829-4d94-9971-25ae9f43646b None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:38.998285 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf4cb9b-a246-4562-902c-77fa74344149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:38.998955 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4cb9b-a246-4562-902c-77fa74344149 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:38.999244 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4cb9b-a246-4562-902c-77fa74344149 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:38.999524 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf4cb9b-a246-4562-902c-77fa74344149 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:39.000313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf4cb9b-a246-4562-902c-77fa74344149 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:53:39.000626 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf4cb9b-a246-4562-902c-77fa74344149 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:39.002203 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf4cb9b-a246-4562-902c-77fa74344149 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:39.014279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 8/15] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:53:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:53:39.015000 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b164641e-a84d-44fc-9e70-32148f7098a9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:39.016626 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b164641e-a84d-44fc-9e70-32148f7098a9 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:39.019637 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf4cb9b-a246-4562-902c-77fa74344149 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:39.114599 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 8/16] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:53:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 06 16:53:39.118378 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf4cb9b-a246-4562-902c-77fa74344149 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:39.120184 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b164641e-a84d-44fc-9e70-32148f7098a9 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:39.122887 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:39.152713 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e198acb-40d8-408f-8fb0-884a2209999c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:39.153592 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e198acb-40d8-408f-8fb0-884a2209999c None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:39.153884 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e198acb-40d8-408f-8fb0-884a2209999c None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:39.154163 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e198acb-40d8-408f-8fb0-884a2209999c None admin] PATH_INFO: `/v3/users/fb49b9dd86cd415bb41412ca15c4d0ba` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:39.155369 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e198acb-40d8-408f-8fb0-884a2209999c None admin] RBAC: Authorizing `identity:get_user(user_id=fb49b9dd86cd415bb41412ca15c4d0ba)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:39.156870 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e198acb-40d8-408f-8fb0-884a2209999c None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:39.158498 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 9/17] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:53:39 2022] GET /identity/v3/users/fb49b9dd86cd415bb41412ca15c4d0ba => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:53:39.163811 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e198acb-40d8-408f-8fb0-884a2209999c None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:39.178502 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:39.217899 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b164641e-a84d-44fc-9e70-32148f7098a9 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:39.255003 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6309c5-e43b-4acc-b302-485f1d7c082e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:39.255836 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6309c5-e43b-4acc-b302-485f1d7c082e None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:39.256135 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6309c5-e43b-4acc-b302-485f1d7c082e None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:39.256411 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6309c5-e43b-4acc-b302-485f1d7c082e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:39.264422 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e198acb-40d8-408f-8fb0-884a2209999c None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:39.275104 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6309c5-e43b-4acc-b302-485f1d7c082e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:39.276606 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6309c5-e43b-4acc-b302-485f1d7c082e None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:39.297218 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a1cb6-e20e-45bf-8781-9a16e216872d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:39.297386 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-8a6309c5-e43b-4acc-b302-485f1d7c082e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:53:39.297386 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:53:39.297386 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:53:39.297386 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:53:39.297386 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:53:39.297386 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:53:39.297386 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:53:39.297386 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:53:39.297386 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:53:39.297386 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:53:39.297386 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:53:39.297386 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:53:39.297386 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:53:39.297386 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:53:39.297386 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:53:39.298155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:53:39.298155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:53:39.298155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:53:39.298155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:53:39.298155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:53:39.298155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:53:39.298155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:53:39.298155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:53:39.298155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:53:39.298155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:53:39.298155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:53:39.298155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:53:39.298155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:53:39.298155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:53:39.298155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:53:39.298155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:53:39.298155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:53:39.298949 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:53:39.298949 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:53:39.298949 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:53:39.298949 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:53:39.299177 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a1cb6-e20e-45bf-8781-9a16e216872d None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:39.299177 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a1cb6-e20e-45bf-8781-9a16e216872d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:39.299298 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 9/18] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:53:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:53:39.299369 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a1cb6-e20e-45bf-8781-9a16e216872d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:39.299773 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a1cb6-e20e-45bf-8781-9a16e216872d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:39.301151 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a1cb6-e20e-45bf-8781-9a16e216872d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:39.302730 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 10/19] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:53:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:53:39.304170 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6309c5-e43b-4acc-b302-485f1d7c082e None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:39.308118 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a1cb6-e20e-45bf-8781-9a16e216872d None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:39.361584 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:39.387118 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:39.405238 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6309c5-e43b-4acc-b302-485f1d7c082e None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:39.406803 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a1cb6-e20e-45bf-8781-9a16e216872d None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:39.438946 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fd8386-1387-4578-bd38-6991eaa5f797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:39.440059 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd8386-1387-4578-bd38-6991eaa5f797 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:39.440349 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd8386-1387-4578-bd38-6991eaa5f797 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:39.440618 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fd8386-1387-4578-bd38-6991eaa5f797 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:39.441341 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd8386-1387-4578-bd38-6991eaa5f797 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:53:39.441503 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6689dd-5d32-4d88-90b6-d66c4bd67118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:39.441646 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd8386-1387-4578-bd38-6991eaa5f797 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:39.442161 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6689dd-5d32-4d88-90b6-d66c4bd67118 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:39.442450 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6689dd-5d32-4d88-90b6-d66c4bd67118 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:39.442730 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6689dd-5d32-4d88-90b6-d66c4bd67118 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:39.443021 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fd8386-1387-4578-bd38-6991eaa5f797 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:39.443563 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6689dd-5d32-4d88-90b6-d66c4bd67118 None admin] RBAC: Adding query filter params (role.id=b41f8a7a58e64f749f2099ca45cc04a7, scope.domain.id=default, user.id=fb49b9dd86cd415bb41412ca15c4d0ba) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:53:39.443905 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6689dd-5d32-4d88-90b6-d66c4bd67118 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:39.445326 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6689dd-5d32-4d88-90b6-d66c4bd67118 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:39.452079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 11/20] 10.222.0.40 () {56 vars in 1322 bytes} [Wed Jul 6 16:53:39 2022] GET /identity/v3/role_assignments?user.id=fb49b9dd86cd415bb41412ca15c4d0ba&scope.domain.id=default&role.id=b41f8a7a58e64f749f2099ca45cc04a7 => generated 233 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:53:39.460636 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 10/21] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:53:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:53:41.862541 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d1c18-cb2f-4b14-88a1-d1eba337a0f2 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:41.863575 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d1c18-cb2f-4b14-88a1-d1eba337a0f2 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:41.864116 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d1c18-cb2f-4b14-88a1-d1eba337a0f2 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:41.865701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 12/22] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:53:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:53:41.871106 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e411dc-827a-42bc-88d3-0000a7a8d09d None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:41.871549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e411dc-827a-42bc-88d3-0000a7a8d09d None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:41.871716 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e411dc-827a-42bc-88d3-0000a7a8d09d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:41.900562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-98e411dc-827a-42bc-88d3-0000a7a8d09d None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:53:41.901790 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-98e411dc-827a-42bc-88d3-0000a7a8d09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:53:41.997551 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 11/23] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:53:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:53:42.032096 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029d1c18-cb2f-4b14-88a1-d1eba337a0f2 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:42.074164 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d31d67-e187-41ff-b2f5-d0c06bb38810 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:42.074842 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d31d67-e187-41ff-b2f5-d0c06bb38810 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:42.075353 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d31d67-e187-41ff-b2f5-d0c06bb38810 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:42.077159 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 12/24] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:53:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:53:42.082696 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e544f0-4a21-472f-805c-87ca87bf3592 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:42.083254 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e544f0-4a21-472f-805c-87ca87bf3592 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:42.083753 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e544f0-4a21-472f-805c-87ca87bf3592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:42.116765 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-34e544f0-4a21-472f-805c-87ca87bf3592 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:53:42.118294 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-34e544f0-4a21-472f-805c-87ca87bf3592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:53:42.217352 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029d1c18-cb2f-4b14-88a1-d1eba337a0f2 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:42.223746 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:42.253016 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 13/25] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:53:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 16:53:42.254614 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af220d40-45ea-4e33-a544-7930ccefcd5b None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:42.255098 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af220d40-45ea-4e33-a544-7930ccefcd5b None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:42.255714 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af220d40-45ea-4e33-a544-7930ccefcd5b None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:42.257556 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 14/26] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:53:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:53:42.264109 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daebdde3-21e9-4a7a-9e9d-6c94d58ab13d None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:42.264696 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daebdde3-21e9-4a7a-9e9d-6c94d58ab13d None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:42.265142 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daebdde3-21e9-4a7a-9e9d-6c94d58ab13d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:42.295595 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-daebdde3-21e9-4a7a-9e9d-6c94d58ab13d None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:53:42.297324 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-daebdde3-21e9-4a7a-9e9d-6c94d58ab13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:53:42.341041 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eedf0e0-45bf-4152-8616-9b266e072e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:42.347837 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eedf0e0-45bf-4152-8616-9b266e072e5c None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:42.348317 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eedf0e0-45bf-4152-8616-9b266e072e5c None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:42.348734 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eedf0e0-45bf-4152-8616-9b266e072e5c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:42.370536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eedf0e0-45bf-4152-8616-9b266e072e5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:53:42.386279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-6eedf0e0-45bf-4152-8616-9b266e072e5c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:53:42.386279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:53:42.386279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:53:42.386279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:53:42.386279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:53:42.386279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:53:42.386279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:53:42.386279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:53:42.386279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:53:42.386279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:53:42.386279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:53:42.386279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:53:42.386279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:53:42.386279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:53:42.386279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:53:42.386279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:53:42.387721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:53:42.387721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:53:42.387721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:53:42.387721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:53:42.387721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:53:42.387721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:53:42.387721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:53:42.387721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:53:42.387721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:53:42.387721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:53:42.387721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:53:42.387721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:53:42.387721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:53:42.387721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:53:42.387721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:53:42.387721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:53:42.388803 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:53:42.388803 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:53:42.388803 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:53:42.388803 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:53:42.388803 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:53:42.388803 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:53:42.388803 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:53:42.388803 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:53:42.388803 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:53:42.388803 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:53:42.388803 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:53:42.388803 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:53:42.391455 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 13/27] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 6 16:53:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:53:42.397013 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eedf0e0-45bf-4152-8616-9b266e072e5c None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:42.399881 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 15/28] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:53:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 16:53:42.401377 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0299d72-79a2-4546-ac0f-618b4fe4c84a None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:42.401696 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:42.402001 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0299d72-79a2-4546-ac0f-618b4fe4c84a None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:42.402412 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0299d72-79a2-4546-ac0f-618b4fe4c84a None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:42.404661 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 16/29] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:53:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:53:42.409975 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0299d72-79a2-4546-ac0f-618b4fe4c84a None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:42.524301 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0299d72-79a2-4546-ac0f-618b4fe4c84a None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:42.531440 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eedf0e0-45bf-4152-8616-9b266e072e5c None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:42.532069 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:42.560306 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9931f02-c329-489e-8b97-8a10dfc25252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:42.561222 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9931f02-c329-489e-8b97-8a10dfc25252 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:42.561941 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9931f02-c329-489e-8b97-8a10dfc25252 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:42.562448 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9931f02-c329-489e-8b97-8a10dfc25252 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:42.563473 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9931f02-c329-489e-8b97-8a10dfc25252 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:53:42.564010 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9931f02-c329-489e-8b97-8a10dfc25252 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:42.565863 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9931f02-c329-489e-8b97-8a10dfc25252 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:42.570115 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875bb81d-c0f4-4d72-a7de-44c62156b5c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:42.570796 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875bb81d-c0f4-4d72-a7de-44c62156b5c7 None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:42.571081 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875bb81d-c0f4-4d72-a7de-44c62156b5c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:42.571356 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875bb81d-c0f4-4d72-a7de-44c62156b5c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:42.572148 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875bb81d-c0f4-4d72-a7de-44c62156b5c7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:42.573535 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875bb81d-c0f4-4d72-a7de-44c62156b5c7 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:42.580737 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 17/30] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:53:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:53:42.582282 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f09c30-cad3-4b31-83b4-57c2009a4f9f None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:42.582578 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f09c30-cad3-4b31-83b4-57c2009a4f9f None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:42.582857 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f09c30-cad3-4b31-83b4-57c2009a4f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:42.599300 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-40f09c30-cad3-4b31-83b4-57c2009a4f9f None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:53:42.600512 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-40f09c30-cad3-4b31-83b4-57c2009a4f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:53:42.675753 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 18/31] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:53:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:53:42.675753 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f09c30-cad3-4b31-83b4-57c2009a4f9f None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:42.796752 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:42.800881 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f09c30-cad3-4b31-83b4-57c2009a4f9f None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:42.816506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 14/32] 10.222.0.40 () {60 vars in 1131 bytes} [Wed Jul 6 16:53:42 2022] POST /identity/v3/roles => generated 219 bytes in 424 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 16:53:42.817459 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875bb81d-c0f4-4d72-a7de-44c62156b5c7 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:42.841637 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07707d2a-003d-42c2-a2fa-16ec01c28e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:42.842720 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07707d2a-003d-42c2-a2fa-16ec01c28e21 None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:42.843424 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07707d2a-003d-42c2-a2fa-16ec01c28e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:42.843927 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07707d2a-003d-42c2-a2fa-16ec01c28e21 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:42.844828 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07707d2a-003d-42c2-a2fa-16ec01c28e21 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:42.846387 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07707d2a-003d-42c2-a2fa-16ec01c28e21 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:42.902274 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:42.951756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875bb81d-c0f4-4d72-a7de-44c62156b5c7 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:42.992494 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6affa64e-ac5b-456c-961c-38e9ce004860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:42.993768 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affa64e-ac5b-456c-961c-38e9ce004860 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:42.994255 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affa64e-ac5b-456c-961c-38e9ce004860 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:42.994730 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6affa64e-ac5b-456c-961c-38e9ce004860 None admin] PATH_INFO: `/v3/users/fb49b9dd86cd415bb41412ca15c4d0ba` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:42.996946 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6affa64e-ac5b-456c-961c-38e9ce004860 None admin] RBAC: Authorizing `identity:get_user(user_id=fb49b9dd86cd415bb41412ca15c4d0ba)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:42.998768 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6affa64e-ac5b-456c-961c-38e9ce004860 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:43.000642 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 15/33] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:53:42 2022] GET /identity/v3/users/fb49b9dd86cd415bb41412ca15c4d0ba => generated 247 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:53:43.001613 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6affa64e-ac5b-456c-961c-38e9ce004860 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:43.069767 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 19/34] 10.222.0.40 () {60 vars in 1131 bytes} [Wed Jul 6 16:53:42 2022] POST /identity/v3/roles => generated 225 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 16:53:43.070975 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07707d2a-003d-42c2-a2fa-16ec01c28e21 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:43.154192 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6affa64e-ac5b-456c-961c-38e9ce004860 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:43.193290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:43.196913 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5994304a-3268-4e1b-8506-07dc857240cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:43.197734 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994304a-3268-4e1b-8506-07dc857240cd None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:43.198017 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994304a-3268-4e1b-8506-07dc857240cd None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:43.198294 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5994304a-3268-4e1b-8506-07dc857240cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:43.211852 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5994304a-3268-4e1b-8506-07dc857240cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:43.213480 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5994304a-3268-4e1b-8506-07dc857240cd None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:43.215170 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07707d2a-003d-42c2-a2fa-16ec01c28e21 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:43.237300 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-5994304a-3268-4e1b-8506-07dc857240cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:53:43.237300 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:53:43.237300 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:53:43.237300 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:53:43.237300 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:53:43.237300 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:53:43.237300 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:53:43.237300 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:53:43.237300 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:53:43.237300 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:53:43.237300 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:53:43.237300 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:53:43.237300 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:53:43.237300 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:53:43.237300 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:53:43.238168 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:53:43.238168 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:53:43.238168 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:53:43.238168 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:53:43.238168 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:53:43.238168 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:53:43.238168 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:53:43.238168 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:53:43.238168 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:53:43.238168 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:53:43.238168 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:53:43.238168 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:53:43.238168 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:53:43.238168 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:53:43.238168 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:53:43.238168 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:53:43.238168 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:53:43.239022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:53:43.239022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:53:43.239022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:53:43.239022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:53:43.239022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 16/35] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:53:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:53:43.241155 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67889b77-316b-4d5b-a20d-28cd8232ca0c None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:43.241485 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67889b77-316b-4d5b-a20d-28cd8232ca0c None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:43.241837 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67889b77-316b-4d5b-a20d-28cd8232ca0c None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:43.243283 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 17/36] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:53:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:53:43.252122 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67889b77-316b-4d5b-a20d-28cd8232ca0c None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:43.264373 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ae2ea-f609-4437-8b1b-223e09be6f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:43.265256 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ae2ea-f609-4437-8b1b-223e09be6f0b None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:43.265589 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ae2ea-f609-4437-8b1b-223e09be6f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:43.265924 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ae2ea-f609-4437-8b1b-223e09be6f0b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:43.267833 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094ae2ea-f609-4437-8b1b-223e09be6f0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:43.269535 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094ae2ea-f609-4437-8b1b-223e09be6f0b None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:43.275029 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 20/37] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:53:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:53:43.275029 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ebdcbb-143c-420c-b9f3-258a14f58eb0 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:43.275029 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ebdcbb-143c-420c-b9f3-258a14f58eb0 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:43.275029 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ebdcbb-143c-420c-b9f3-258a14f58eb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:43.297776 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-d3ebdcbb-143c-420c-b9f3-258a14f58eb0 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:53:43.299468 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d3ebdcbb-143c-420c-b9f3-258a14f58eb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:53:43.323797 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:43.383830 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 21/38] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:53:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:53:43.384715 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ebdcbb-143c-420c-b9f3-258a14f58eb0 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:43.406421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67889b77-316b-4d5b-a20d-28cd8232ca0c None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:43.497670 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ebdcbb-143c-420c-b9f3-258a14f58eb0 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:43.533844 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995f5eb6-225a-442a-8193-d69a2de3d8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:43.534728 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f5eb6-225a-442a-8193-d69a2de3d8e2 None admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:43.535020 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f5eb6-225a-442a-8193-d69a2de3d8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:43.535303 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995f5eb6-225a-442a-8193-d69a2de3d8e2 None admin] PATH_INFO: `/v3/domains/default/users/fb49b9dd86cd415bb41412ca15c4d0ba/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:43.537703 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995f5eb6-225a-442a-8193-d69a2de3d8e2 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=fb49b9dd86cd415bb41412ca15c4d0ba, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:43.539174 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995f5eb6-225a-442a-8193-d69a2de3d8e2 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:43.547657 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c393ec-54b1-497f-ad31-cd2ad2dd38ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:43.548357 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c393ec-54b1-497f-ad31-cd2ad2dd38ed None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:43.548649 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c393ec-54b1-497f-ad31-cd2ad2dd38ed None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:43.548937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c393ec-54b1-497f-ad31-cd2ad2dd38ed None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:43.549673 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c393ec-54b1-497f-ad31-cd2ad2dd38ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:53:43.549985 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c393ec-54b1-497f-ad31-cd2ad2dd38ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:43.551361 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c393ec-54b1-497f-ad31-cd2ad2dd38ed None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:43.569281 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 18/39] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:53:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:53:43.570158 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c393ec-54b1-497f-ad31-cd2ad2dd38ed None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:43.616953 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:43.642056 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 22/40] 10.222.0.40 () {58 vars in 1361 bytes} [Wed Jul 6 16:53:43 2022] PUT /identity/v3/domains/default/users/fb49b9dd86cd415bb41412ca15c4d0ba/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 16:53:43.643369 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb8d2c-e595-42e5-9ab3-1ca617abd551 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:43.643658 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb8d2c-e595-42e5-9ab3-1ca617abd551 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:43.643963 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb8d2c-e595-42e5-9ab3-1ca617abd551 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:43.646428 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 23/41] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:53:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:53:43.646542 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb8d2c-e595-42e5-9ab3-1ca617abd551 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:43.668181 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:43.687983 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c393ec-54b1-497f-ad31-cd2ad2dd38ed None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:43.732320 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442294b2-73ca-406a-9b57-aa3b950de73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:43.733237 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442294b2-73ca-406a-9b57-aa3b950de73b None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:43.733367 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442294b2-73ca-406a-9b57-aa3b950de73b None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:43.733746 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442294b2-73ca-406a-9b57-aa3b950de73b None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:43.734416 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442294b2-73ca-406a-9b57-aa3b950de73b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:43.735871 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442294b2-73ca-406a-9b57-aa3b950de73b None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:43.770143 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb8d2c-e595-42e5-9ab3-1ca617abd551 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:43.807816 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da9cd7-da38-42a0-a80b-6baaa3b56cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:43.808491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da9cd7-da38-42a0-a80b-6baaa3b56cbc None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:43.808780 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da9cd7-da38-42a0-a80b-6baaa3b56cbc None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:43.809066 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da9cd7-da38-42a0-a80b-6baaa3b56cbc None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:43.809778 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8da9cd7-da38-42a0-a80b-6baaa3b56cbc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:43.811248 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8da9cd7-da38-42a0-a80b-6baaa3b56cbc None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:43.819877 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 19/42] 10.222.0.40 () {60 vars in 1131 bytes} [Wed Jul 6 16:53:43 2022] POST /identity/v3/roles => generated 223 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 16:53:43.821067 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0276a534-60b9-4880-ab0c-1676e2250bb9 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:43.821365 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0276a534-60b9-4880-ab0c-1676e2250bb9 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:43.827844 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0276a534-60b9-4880-ab0c-1676e2250bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:43.849650 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-0276a534-60b9-4880-ab0c-1676e2250bb9 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:53:43.851350 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-0276a534-60b9-4880-ab0c-1676e2250bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:53:43.905807 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 24/43] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Jul 6 16:53:43 2022] POST /identity/v3/projects => generated 301 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 16:53:43.937359 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 20/44] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:53:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:53:43.960563 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da9cd7-da38-42a0-a80b-6baaa3b56cbc None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:44.040284 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:44.080372 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da9cd7-da38-42a0-a80b-6baaa3b56cbc None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:44.116840 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cd4a78-399d-4e9f-a6d7-4c1ad47758f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:44.117880 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd4a78-399d-4e9f-a6d7-4c1ad47758f6 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:44.118348 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd4a78-399d-4e9f-a6d7-4c1ad47758f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:44.118714 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cd4a78-399d-4e9f-a6d7-4c1ad47758f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:44.119940 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cd4a78-399d-4e9f-a6d7-4c1ad47758f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:44.121440 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cd4a78-399d-4e9f-a6d7-4c1ad47758f6 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:44.123075 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 25/45] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:53:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:53:44.128737 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0276a534-60b9-4880-ab0c-1676e2250bb9 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:44.165752 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:44.229954 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0276a534-60b9-4880-ab0c-1676e2250bb9 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:44.263062 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b62395-2440-4ca1-be66-148592ec6763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:44.263721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b62395-2440-4ca1-be66-148592ec6763 None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:44.264031 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b62395-2440-4ca1-be66-148592ec6763 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:44.264309 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b62395-2440-4ca1-be66-148592ec6763 None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:44.265025 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b62395-2440-4ca1-be66-148592ec6763 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:44.266483 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b62395-2440-4ca1-be66-148592ec6763 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:44.346248 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 21/46] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Jul 6 16:53:44 2022] POST /identity/v3/projects => generated 312 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 16:53:45.647261 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853cfc37-3044-4c3d-82ac-9a47598ee07d None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:45.647820 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853cfc37-3044-4c3d-82ac-9a47598ee07d None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:45.647904 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853cfc37-3044-4c3d-82ac-9a47598ee07d None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:45.649217 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 26/47] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:53:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:53:45.655058 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8cde40-dbb7-4b3d-a82a-6a12e038e209 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:45.655477 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8cde40-dbb7-4b3d-a82a-6a12e038e209 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:45.655629 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8cde40-dbb7-4b3d-a82a-6a12e038e209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:45.672655 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-6d8cde40-dbb7-4b3d-a82a-6a12e038e209 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:53:45.673830 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-6d8cde40-dbb7-4b3d-a82a-6a12e038e209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:53:45.741413 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 22/48] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:53:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:53:45.763706 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853cfc37-3044-4c3d-82ac-9a47598ee07d None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:45.864432 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853cfc37-3044-4c3d-82ac-9a47598ee07d None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:45.897392 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8fc45d-a33a-4bf1-b8c6-64cfac1625bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:45.898116 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8fc45d-a33a-4bf1-b8c6-64cfac1625bf None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:45.898403 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8fc45d-a33a-4bf1-b8c6-64cfac1625bf None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:45.898682 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8fc45d-a33a-4bf1-b8c6-64cfac1625bf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:45.902237 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:46.015084 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8fc45d-a33a-4bf1-b8c6-64cfac1625bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:53:46.024248 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-4c8fc45d-a33a-4bf1-b8c6-64cfac1625bf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:53:46.024248 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:53:46.024248 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:53:46.024248 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:53:46.024248 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:53:46.024248 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:53:46.024248 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:53:46.024248 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:53:46.024248 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:53:46.024248 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:53:46.024248 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:53:46.024248 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:53:46.024248 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:53:46.024248 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:53:46.024248 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:53:46.024248 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:53:46.025076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:53:46.025076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:53:46.025076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:53:46.025076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:53:46.025076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:53:46.025076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:53:46.025076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:53:46.025076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:53:46.025076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:53:46.025076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:53:46.025076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:53:46.025076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:53:46.025076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:53:46.025076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:53:46.025076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:53:46.025076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:53:46.025804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:53:46.025804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:53:46.025804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:53:46.025804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:53:46.025804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:53:46.025804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:53:46.025804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:53:46.025804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:53:46.025804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:53:46.025804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:53:46.025804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:53:46.025804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:53:46.025804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 27/49] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 6 16:53:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:53:46.032997 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8cde40-dbb7-4b3d-a82a-6a12e038e209 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:46.051916 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:46.130414 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8cde40-dbb7-4b3d-a82a-6a12e038e209 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:46.164241 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707816eb-8988-41de-a6ba-8a6c80412037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:46.164915 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707816eb-8988-41de-a6ba-8a6c80412037 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:46.165203 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707816eb-8988-41de-a6ba-8a6c80412037 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:46.165484 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-707816eb-8988-41de-a6ba-8a6c80412037 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:46.166234 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707816eb-8988-41de-a6ba-8a6c80412037 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:53:46.166548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707816eb-8988-41de-a6ba-8a6c80412037 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:46.167970 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-707816eb-8988-41de-a6ba-8a6c80412037 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:46.175961 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 23/50] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:53:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:53:46.181393 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8fc45d-a33a-4bf1-b8c6-64cfac1625bf None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:46.280388 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8fc45d-a33a-4bf1-b8c6-64cfac1625bf None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:46.313147 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871495be-83de-4bf1-ab80-c4aba771b816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:46.314023 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871495be-83de-4bf1-ab80-c4aba771b816 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:46.314312 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871495be-83de-4bf1-ab80-c4aba771b816 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:46.314595 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871495be-83de-4bf1-ab80-c4aba771b816 None admin] PATH_INFO: `/v3/users/fb49b9dd86cd415bb41412ca15c4d0ba` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:46.315819 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871495be-83de-4bf1-ab80-c4aba771b816 None admin] RBAC: Authorizing `identity:get_user(user_id=fb49b9dd86cd415bb41412ca15c4d0ba)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:46.317276 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871495be-83de-4bf1-ab80-c4aba771b816 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:46.318913 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 28/51] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:53:46 2022] GET /identity/v3/users/fb49b9dd86cd415bb41412ca15c4d0ba => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:53:46.324992 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707816eb-8988-41de-a6ba-8a6c80412037 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:46.369251 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910d0d6-cfbe-4a9e-abad-0a296c288e6b None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:46.369542 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910d0d6-cfbe-4a9e-abad-0a296c288e6b None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:46.369828 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910d0d6-cfbe-4a9e-abad-0a296c288e6b None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:46.371684 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 29/52] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:53:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 16:53:46.376464 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de1a197-38ae-4d23-871e-3c5739779852 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:46.376755 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de1a197-38ae-4d23-871e-3c5739779852 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:46.377035 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de1a197-38ae-4d23-871e-3c5739779852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:46.393711 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-4de1a197-38ae-4d23-871e-3c5739779852 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:53:46.394896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-4de1a197-38ae-4d23-871e-3c5739779852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:53:46.401817 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:46.422487 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-707816eb-8988-41de-a6ba-8a6c80412037 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:46.453422 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:46.456290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f5e7e6-fbbf-40d2-9e27-79b8d22907d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:46.457100 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5e7e6-fbbf-40d2-9e27-79b8d22907d7 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:46.457389 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5e7e6-fbbf-40d2-9e27-79b8d22907d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:46.457670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f5e7e6-fbbf-40d2-9e27-79b8d22907d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:46.458845 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f5e7e6-fbbf-40d2-9e27-79b8d22907d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:46.460390 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f5e7e6-fbbf-40d2-9e27-79b8d22907d7 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:46.461151 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 30/53] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:53:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 16:53:46.461977 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 24/54] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:53:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:53:46.467417 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de1a197-38ae-4d23-871e-3c5739779852 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:46.483845 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f5e7e6-fbbf-40d2-9e27-79b8d22907d7 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:46.565396 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de1a197-38ae-4d23-871e-3c5739779852 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:46.581837 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f5e7e6-fbbf-40d2-9e27-79b8d22907d7 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:46.598835 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e8275-f8a7-4d04-8b23-6db4e4933747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:46.599561 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e8275-f8a7-4d04-8b23-6db4e4933747 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:46.599880 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e8275-f8a7-4d04-8b23-6db4e4933747 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:46.600158 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361e8275-f8a7-4d04-8b23-6db4e4933747 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:46.600993 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361e8275-f8a7-4d04-8b23-6db4e4933747 None admin] RBAC: Adding query filter params (role.id=b41f8a7a58e64f749f2099ca45cc04a7, scope.domain.id=default, user.id=fb49b9dd86cd415bb41412ca15c4d0ba) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:53:46.601304 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361e8275-f8a7-4d04-8b23-6db4e4933747 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:46.602756 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361e8275-f8a7-4d04-8b23-6db4e4933747 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:46.611277 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 31/55] 10.222.0.40 () {56 vars in 1322 bytes} [Wed Jul 6 16:53:46 2022] GET /identity/v3/role_assignments?user.id=fb49b9dd86cd415bb41412ca15c4d0ba&scope.domain.id=default&role.id=b41f8a7a58e64f749f2099ca45cc04a7 => generated 530 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:53:46.615463 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c39015c-d785-4277-8a99-541abc2c6e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:46.616311 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39015c-d785-4277-8a99-541abc2c6e6a None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:46.616613 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39015c-d785-4277-8a99-541abc2c6e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:46.616898 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c39015c-d785-4277-8a99-541abc2c6e6a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:46.618143 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c39015c-d785-4277-8a99-541abc2c6e6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:46.619633 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c39015c-d785-4277-8a99-541abc2c6e6a None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:46.621997 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 25/56] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:53:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:53:46.627929 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e8275-f8a7-4d04-8b23-6db4e4933747 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:46.691374 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:46.738934 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361e8275-f8a7-4d04-8b23-6db4e4933747 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:46.774949 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f313556-c301-42fe-b77f-f485d4e937dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:46.775618 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f313556-c301-42fe-b77f-f485d4e937dc None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:46.775939 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f313556-c301-42fe-b77f-f485d4e937dc None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:46.776226 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f313556-c301-42fe-b77f-f485d4e937dc None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:46.776936 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f313556-c301-42fe-b77f-f485d4e937dc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:46.778393 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f313556-c301-42fe-b77f-f485d4e937dc None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:46.865503 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 32/57] 10.222.0.40 () {60 vars in 1140 bytes} [Wed Jul 6 16:53:46 2022] POST /identity/v3/projects => generated 298 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 16:53:48.881759 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9cf4a3-8221-4385-9751-302a22832632 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:48.882266 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9cf4a3-8221-4385-9751-302a22832632 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:48.882350 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9cf4a3-8221-4385-9751-302a22832632 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:48.883792 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 26/58] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:53:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:53:48.889308 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b94fd1-5b1f-4fe0-b8ad-bb2c3482d482 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:48.889747 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b94fd1-5b1f-4fe0-b8ad-bb2c3482d482 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:48.889881 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b94fd1-5b1f-4fe0-b8ad-bb2c3482d482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:48.907127 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-55b94fd1-5b1f-4fe0-b8ad-bb2c3482d482 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:53:48.908355 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-55b94fd1-5b1f-4fe0-b8ad-bb2c3482d482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:53:48.972396 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 33/59] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:53:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 16:53:48.996002 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9cf4a3-8221-4385-9751-302a22832632 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:49.012170 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:49.094609 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9cf4a3-8221-4385-9751-302a22832632 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:49.126396 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8748e2e9-b767-430f-ac50-83390720408c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:49.127179 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748e2e9-b767-430f-ac50-83390720408c None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:49.127469 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748e2e9-b767-430f-ac50-83390720408c None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:49.127769 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8748e2e9-b767-430f-ac50-83390720408c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:49.128942 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8748e2e9-b767-430f-ac50-83390720408c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:49.130382 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8748e2e9-b767-430f-ac50-83390720408c None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:49.132003 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 27/60] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:53:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:53:49.137687 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b94fd1-5b1f-4fe0-b8ad-bb2c3482d482 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:49.178438 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:49.233712 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b94fd1-5b1f-4fe0-b8ad-bb2c3482d482 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:49.266874 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9fe4da-dd8c-4850-882d-22e60f1a0ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:49.267473 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9fe4da-dd8c-4850-882d-22e60f1a0ef7 None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:49.267798 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9fe4da-dd8c-4850-882d-22e60f1a0ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:49.268097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9fe4da-dd8c-4850-882d-22e60f1a0ef7 None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:49.268854 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9fe4da-dd8c-4850-882d-22e60f1a0ef7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:49.270288 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9fe4da-dd8c-4850-882d-22e60f1a0ef7 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:49.394672 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 34/61] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 6 16:53:49 2022] POST /identity/v3/users => generated 275 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 16:53:52.467902 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d51eb6-d69b-417a-b50d-f82cd15ed3cf None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:52.468456 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d51eb6-d69b-417a-b50d-f82cd15ed3cf None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:52.468456 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d51eb6-d69b-417a-b50d-f82cd15ed3cf None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:52.469855 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 28/62] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:53:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:53:52.475575 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c10d9-5326-4bab-8ab1-41044558adfb None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:52.476079 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c10d9-5326-4bab-8ab1-41044558adfb None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:52.476272 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c10d9-5326-4bab-8ab1-41044558adfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:52.504845 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-876c10d9-5326-4bab-8ab1-41044558adfb None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:53:52.506087 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-876c10d9-5326-4bab-8ab1-41044558adfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:53:52.602313 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 35/63] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:53:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:53:52.629090 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d51eb6-d69b-417a-b50d-f82cd15ed3cf None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:52.642198 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da486e-7d6a-47de-8fbe-fce6ea23e8b6 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:52.642491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da486e-7d6a-47de-8fbe-fce6ea23e8b6 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:52.642781 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7da486e-7d6a-47de-8fbe-fce6ea23e8b6 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:52.644392 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 36/64] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:53:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 16:53:52.649794 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57673248-02dc-467d-899f-696a404b9a0a None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:52.650092 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57673248-02dc-467d-899f-696a404b9a0a None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:52.650382 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57673248-02dc-467d-899f-696a404b9a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:52.681793 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-57673248-02dc-467d-899f-696a404b9a0a None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:53:52.683383 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-57673248-02dc-467d-899f-696a404b9a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:53:52.724160 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:52.757518 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:52.780145 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d51eb6-d69b-417a-b50d-f82cd15ed3cf None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:52.806162 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 37/65] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:53:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 16:53:52.807509 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14636d9-71f4-4a5f-9163-a51a2700f870 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:52.807833 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14636d9-71f4-4a5f-9163-a51a2700f870 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:52.808122 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14636d9-71f4-4a5f-9163-a51a2700f870 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:52.809719 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 38/66] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:53:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:53:52.814935 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a8489-f104-4ebf-87d2-c7f18ea3712b None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:52.815392 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a8489-f104-4ebf-87d2-c7f18ea3712b None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:52.815920 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1a8489-f104-4ebf-87d2-c7f18ea3712b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:52.833612 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c0025-018c-4844-906e-ab6f2749697a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:52.834516 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c0025-018c-4844-906e-ab6f2749697a None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:52.834814 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c0025-018c-4844-906e-ab6f2749697a None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:52.835094 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c0025-018c-4844-906e-ab6f2749697a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:52.840846 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-2e1a8489-f104-4ebf-87d2-c7f18ea3712b None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:53:52.842300 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-2e1a8489-f104-4ebf-87d2-c7f18ea3712b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:53:52.846827 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1c0025-018c-4844-906e-ab6f2749697a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:53:52.855024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-0f1c0025-018c-4844-906e-ab6f2749697a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 16:53:52.855024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:53:52.855024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:53:52.855024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:53:52.855024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:53:52.855024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:53:52.855024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:53:52.855024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:53:52.855024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:53:52.855024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:53:52.855024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:53:52.855024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:53:52.855024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:53:52.855024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:53:52.855024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:53:52.855024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:53:52.856464 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:53:52.856464 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:53:52.856464 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:53:52.856464 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:53:52.856464 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:53:52.856464 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:53:52.856464 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:53:52.856464 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:53:52.856464 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:53:52.856464 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:53:52.856464 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:53:52.856464 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:53:52.856464 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:53:52.856464 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:53:52.856464 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:53:52.856464 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:53:52.857208 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:53:52.857208 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:53:52.857208 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:53:52.857208 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:53:52.857208 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:53:52.857208 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:53:52.857208 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:53:52.857208 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:53:52.857208 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:53:52.857208 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:53:52.857208 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 16:53:52.857208 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:53:52.859699 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 29/67] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:53:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:53:52.860839 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c0025-018c-4844-906e-ab6f2749697a None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:52.935987 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 39/68] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:53:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:53:52.937007 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a8489-f104-4ebf-87d2-c7f18ea3712b None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:52.981633 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c0025-018c-4844-906e-ab6f2749697a None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:53.023011 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3651ae0-755c-465a-b9f3-8cc23b9d48ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:53.023844 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3651ae0-755c-465a-b9f3-8cc23b9d48ed None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:53.024137 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3651ae0-755c-465a-b9f3-8cc23b9d48ed None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:53.024416 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3651ae0-755c-465a-b9f3-8cc23b9d48ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:53.036103 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3651ae0-755c-465a-b9f3-8cc23b9d48ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:53:53.039048 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:53.044425 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-a3651ae0-755c-465a-b9f3-8cc23b9d48ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:53:53.044425 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:53:53.044425 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:53:53.044425 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:53:53.044425 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:53:53.044425 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:53:53.044425 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:53:53.044425 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:53:53.044425 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:53:53.044425 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:53:53.044425 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:53:53.044425 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:53:53.044425 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:53:53.044425 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:53:53.044425 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:53:53.044425 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:53:53.045258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:53:53.045258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:53:53.045258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:53:53.045258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:53:53.045258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:53:53.045258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:53:53.045258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:53:53.045258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:53:53.045258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:53:53.045258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:53:53.045258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:53:53.045258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:53:53.045258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:53:53.045258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:53:53.045258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:53:53.045258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:53:53.046090 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:53:53.046090 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:53:53.046090 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:53:53.046090 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:53:53.046090 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:53:53.046090 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:53:53.046090 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:53:53.046090 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:53:53.046090 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:53:53.046090 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:53:53.046090 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:53:53.046090 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:53:53.046090 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 30/69] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 6 16:53:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:53:53.046705 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3651ae0-755c-465a-b9f3-8cc23b9d48ed None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:53.050315 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:53.061615 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1a8489-f104-4ebf-87d2-c7f18ea3712b None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:53.101740 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96a0cdf-dc33-4b92-8dd8-1d1b617ab9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:53.102829 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a0cdf-dc33-4b92-8dd8-1d1b617ab9da None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:53.103257 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a0cdf-dc33-4b92-8dd8-1d1b617ab9da None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:53.103703 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96a0cdf-dc33-4b92-8dd8-1d1b617ab9da None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:53.104641 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96a0cdf-dc33-4b92-8dd8-1d1b617ab9da None admin] RBAC: Adding query filter params (name=member) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:53:53.105132 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96a0cdf-dc33-4b92-8dd8-1d1b617ab9da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:53.107186 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96a0cdf-dc33-4b92-8dd8-1d1b617ab9da None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:53.115654 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 40/70] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 6 16:53:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:53:53.116942 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70abfa80-3741-49bb-b7c9-ecf303c93ee2 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:53.117231 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70abfa80-3741-49bb-b7c9-ecf303c93ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:53.117510 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70abfa80-3741-49bb-b7c9-ecf303c93ee2 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:53.118921 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 41/71] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:53:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:53:53.123829 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70abfa80-3741-49bb-b7c9-ecf303c93ee2 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:53.159405 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3651ae0-755c-465a-b9f3-8cc23b9d48ed None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:53.196091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e760cf7-d016-4b8b-8e09-e378b867374b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:53.196914 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e760cf7-d016-4b8b-8e09-e378b867374b None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:53.197203 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e760cf7-d016-4b8b-8e09-e378b867374b None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:53.197485 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e760cf7-d016-4b8b-8e09-e378b867374b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:53.208077 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e760cf7-d016-4b8b-8e09-e378b867374b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:53:53.216184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-5e760cf7-d016-4b8b-8e09-e378b867374b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 16:53:53.216184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:53:53.216184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:53:53.216184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:53:53.216184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:53:53.216184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:53:53.216184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:53:53.216184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:53:53.216184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:53:53.216184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:53:53.216184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:53:53.216184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:53:53.216184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:53:53.216184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:53:53.216184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:53:53.216184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:53:53.217045 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:53:53.217045 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:53:53.217045 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:53:53.217045 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:53:53.217045 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:53:53.217045 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:53:53.217045 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:53:53.217045 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:53:53.217045 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:53:53.217045 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:53:53.217045 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:53:53.217045 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:53:53.217045 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:53:53.217045 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:53:53.217045 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:53:53.217045 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:53:53.217790 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:53:53.217790 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:53:53.217790 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:53:53.217790 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:53:53.217790 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:53:53.217790 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:53:53.217790 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:53:53.217790 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:53:53.217790 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:53:53.217790 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:53:53.217790 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 16:53:53.217790 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:53:53.217790 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 31/72] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 6 16:53:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:53:53.218407 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e760cf7-d016-4b8b-8e09-e378b867374b None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:53.242018 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70abfa80-3741-49bb-b7c9-ecf303c93ee2 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:53.276885 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807ca0b4-b8ac-4c5c-bda0-c11f7f1eeae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:53.277598 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ca0b4-b8ac-4c5c-bda0-c11f7f1eeae6 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:53.277886 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ca0b4-b8ac-4c5c-bda0-c11f7f1eeae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:53.278162 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807ca0b4-b8ac-4c5c-bda0-c11f7f1eeae6 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:53.278902 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807ca0b4-b8ac-4c5c-bda0-c11f7f1eeae6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:53:53.279207 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807ca0b4-b8ac-4c5c-bda0-c11f7f1eeae6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:53.280634 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807ca0b4-b8ac-4c5c-bda0-c11f7f1eeae6 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:53.284014 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:53.288607 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 42/73] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:53:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:53:53.289794 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e6aff-1486-438d-9a06-f9ae6a98832a None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:53.290084 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e6aff-1486-438d-9a06-f9ae6a98832a None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:53.290423 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294e6aff-1486-438d-9a06-f9ae6a98832a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:53.306116 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-294e6aff-1486-438d-9a06-f9ae6a98832a None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:53:53.311541 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-294e6aff-1486-438d-9a06-f9ae6a98832a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:53:53.311541 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:53.320169 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e760cf7-d016-4b8b-8e09-e378b867374b None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:53.354468 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37fc85e-8860-4064-91ad-565f41ac6062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:53.355920 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37fc85e-8860-4064-91ad-565f41ac6062 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:53.356216 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37fc85e-8860-4064-91ad-565f41ac6062 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:53.356492 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37fc85e-8860-4064-91ad-565f41ac6062 None admin] PATH_INFO: `/v3/users/8ad29aae43d24936a5dbec7f54883329` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:53.369479 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37fc85e-8860-4064-91ad-565f41ac6062 None admin] RBAC: Authorizing `identity:get_user(user_id=8ad29aae43d24936a5dbec7f54883329)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:53.371486 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37fc85e-8860-4064-91ad-565f41ac6062 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:53.373165 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 32/74] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:53:53 2022] GET /identity/v3/users/8ad29aae43d24936a5dbec7f54883329 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:53:53.374355 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c04de-4161-4dfc-9e8b-a45f7dea8bef None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:53.374646 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c04de-4161-4dfc-9e8b-a45f7dea8bef None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:53.374927 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c04de-4161-4dfc-9e8b-a45f7dea8bef None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:53.376175 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 33/75] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:53:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:53:53.376875 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c04de-4161-4dfc-9e8b-a45f7dea8bef None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:53.381049 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 43/76] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:53:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:53:53.381877 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e6aff-1486-438d-9a06-f9ae6a98832a None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:53.484334 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294e6aff-1486-438d-9a06-f9ae6a98832a None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:53.486932 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c04de-4161-4dfc-9e8b-a45f7dea8bef None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:53.507139 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:53.518740 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d48ec8-8356-4cca-a4f0-5c4828fffc75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:53.519612 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d48ec8-8356-4cca-a4f0-5c4828fffc75 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:53.519924 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d48ec8-8356-4cca-a4f0-5c4828fffc75 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:53.520201 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d48ec8-8356-4cca-a4f0-5c4828fffc75 None admin] PATH_INFO: `/v3/users/fb49b9dd86cd415bb41412ca15c4d0ba` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:53.520727 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f48889d-11ee-48bf-93b4-350187cb651d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:53.521639 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d48ec8-8356-4cca-a4f0-5c4828fffc75 None admin] RBAC: Authorizing `identity:get_user(user_id=fb49b9dd86cd415bb41412ca15c4d0ba)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:53.521742 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f48889d-11ee-48bf-93b4-350187cb651d None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:53.521944 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f48889d-11ee-48bf-93b4-350187cb651d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:53.522224 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f48889d-11ee-48bf-93b4-350187cb651d None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:53.523006 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f48889d-11ee-48bf-93b4-350187cb651d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:53:53.523094 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d48ec8-8356-4cca-a4f0-5c4828fffc75 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:53.523319 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f48889d-11ee-48bf-93b4-350187cb651d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:53.524725 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f48889d-11ee-48bf-93b4-350187cb651d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:53.524813 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 44/77] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:53:53 2022] GET /identity/v3/users/fb49b9dd86cd415bb41412ca15c4d0ba => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:53:53.528554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d48ec8-8356-4cca-a4f0-5c4828fffc75 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:53.537543 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 34/78] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:53:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:53:53.537543 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5841312c-7b5b-4277-8b61-a23d505af0b9 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:53.537543 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5841312c-7b5b-4277-8b61-a23d505af0b9 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:53.537543 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5841312c-7b5b-4277-8b61-a23d505af0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:53.539278 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:53.552318 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-5841312c-7b5b-4277-8b61-a23d505af0b9 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:53:53.553575 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-5841312c-7b5b-4277-8b61-a23d505af0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:53:53.619222 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 35/79] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:53:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:53:53.620186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5841312c-7b5b-4277-8b61-a23d505af0b9 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:53.629358 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d48ec8-8356-4cca-a4f0-5c4828fffc75 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:53.664622 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3091fd45-49f4-49ce-9058-a8c9a38b257d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:53.665272 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3091fd45-49f4-49ce-9058-a8c9a38b257d None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:53.665558 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3091fd45-49f4-49ce-9058-a8c9a38b257d None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:53.665836 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3091fd45-49f4-49ce-9058-a8c9a38b257d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:53.717885 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5841312c-7b5b-4277-8b61-a23d505af0b9 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:53.731337 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:53.733495 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3091fd45-49f4-49ce-9058-a8c9a38b257d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:53.734981 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3091fd45-49f4-49ce-9058-a8c9a38b257d None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:53.744610 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:53.753105 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3e2412-7f5d-4d38-a591-18cfbac36792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:53.753904 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e2412-7f5d-4d38-a591-18cfbac36792 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:53.754192 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e2412-7f5d-4d38-a591-18cfbac36792 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:53.754471 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3e2412-7f5d-4d38-a591-18cfbac36792 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:53.755653 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3e2412-7f5d-4d38-a591-18cfbac36792 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:53.757131 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3e2412-7f5d-4d38-a591-18cfbac36792 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:53.758805 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 36/80] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:53:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:53:53.759518 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3e2412-7f5d-4d38-a591-18cfbac36792 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:53.833318 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 45/81] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:53:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 16:53:53.834172 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3091fd45-49f4-49ce-9058-a8c9a38b257d None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:53.859395 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3e2412-7f5d-4d38-a591-18cfbac36792 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:53.892703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a334d2-a7ec-441a-aa7c-02cecc880358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:53.893351 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a334d2-a7ec-441a-aa7c-02cecc880358 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:53.893639 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a334d2-a7ec-441a-aa7c-02cecc880358 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:53.893915 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a334d2-a7ec-441a-aa7c-02cecc880358 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:53.931094 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3091fd45-49f4-49ce-9058-a8c9a38b257d None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:53.940073 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:53.942083 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:53.960243 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a334d2-a7ec-441a-aa7c-02cecc880358 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:53.961803 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a334d2-a7ec-441a-aa7c-02cecc880358 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:53.963419 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c7b30e-27cd-486f-bf9d-5f7dc42709ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:53.964323 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7b30e-27cd-486f-bf9d-5f7dc42709ec None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:53.965130 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7b30e-27cd-486f-bf9d-5f7dc42709ec None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:53.965423 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c7b30e-27cd-486f-bf9d-5f7dc42709ec None admin] PATH_INFO: `/v3/users/8ad29aae43d24936a5dbec7f54883329` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:53.966609 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c7b30e-27cd-486f-bf9d-5f7dc42709ec None admin] RBAC: Authorizing `identity:get_user(user_id=8ad29aae43d24936a5dbec7f54883329)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:53.968075 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c7b30e-27cd-486f-bf9d-5f7dc42709ec None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:53.969915 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 46/82] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:53:53 2022] GET /identity/v3/users/8ad29aae43d24936a5dbec7f54883329 => generated 275 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:53:53.970475 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c7b30e-27cd-486f-bf9d-5f7dc42709ec None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:54.062884 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 37/83] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:53:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 16:53:54.063720 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a334d2-a7ec-441a-aa7c-02cecc880358 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:54.067829 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c7b30e-27cd-486f-bf9d-5f7dc42709ec None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:54.101590 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d9e209-573d-4753-b174-9c73b9796bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:54.102409 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9e209-573d-4753-b174-9c73b9796bed None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:54.102693 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9e209-573d-4753-b174-9c73b9796bed None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:54.102968 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9e209-573d-4753-b174-9c73b9796bed None admin] PATH_INFO: `/v3/roles/member` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:54.113624 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d9e209-573d-4753-b174-9c73b9796bed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:53:54.121363 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-87d9e209-573d-4753-b174-9c73b9796bed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 16:53:54.121363 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:53:54.121363 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:53:54.121363 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:53:54.121363 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:53:54.121363 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:53:54.121363 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:53:54.121363 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:53:54.121363 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:53:54.121363 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:53:54.121363 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:53:54.121363 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:53:54.121363 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:53:54.121363 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:53:54.121363 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:53:54.121363 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:53:54.122213 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:53:54.122213 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:53:54.122213 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:53:54.122213 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:53:54.122213 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:53:54.122213 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:53:54.122213 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:53:54.122213 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:53:54.122213 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:53:54.122213 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:53:54.122213 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:53:54.122213 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:53:54.122213 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:53:54.122213 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:53:54.122213 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:53:54.122213 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:53:54.122978 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:53:54.122978 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:53:54.122978 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:53:54.122978 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:53:54.122978 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:53:54.122978 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:53:54.122978 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:53:54.122978 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:53:54.122978 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:53:54.122978 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:53:54.122978 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 16:53:54.122978 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:53:54.122978 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 47/84] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:53:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:53:54.123631 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d9e209-573d-4753-b174-9c73b9796bed None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:54.146522 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:54.162512 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:54.165512 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a334d2-a7ec-441a-aa7c-02cecc880358 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:54.197972 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949254b3-bd56-4712-89d7-631fd8897a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:54.198643 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949254b3-bd56-4712-89d7-631fd8897a1d None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:54.199434 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949254b3-bd56-4712-89d7-631fd8897a1d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:54.199748 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949254b3-bd56-4712-89d7-631fd8897a1d None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:54.200479 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949254b3-bd56-4712-89d7-631fd8897a1d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:54.201952 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949254b3-bd56-4712-89d7-631fd8897a1d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:54.225875 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d9e209-573d-4753-b174-9c73b9796bed None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:54.258063 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc397958-9e47-4d85-885a-cc95b9a779b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:54.258914 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc397958-9e47-4d85-885a-cc95b9a779b5 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:54.259198 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc397958-9e47-4d85-885a-cc95b9a779b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:54.259472 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc397958-9e47-4d85-885a-cc95b9a779b5 None admin] PATH_INFO: `/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:54.260676 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc397958-9e47-4d85-885a-cc95b9a779b5 None admin] RBAC: Authorizing `identity:get_project(project_id=49b1ed5ea8fc417f8247e689e804e6f1)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:54.262636 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc397958-9e47-4d85-885a-cc95b9a779b5 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:54.264349 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 48/85] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:53:54 2022] GET /identity/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1 => generated 298 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:53:54.265109 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc397958-9e47-4d85-885a-cc95b9a779b5 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:54.304181 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 38/86] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 6 16:53:54 2022] POST /identity/v3/users => generated 289 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 16:53:54.305108 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949254b3-bd56-4712-89d7-631fd8897a1d None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:54.372181 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc397958-9e47-4d85-885a-cc95b9a779b5 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:54.374681 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:54.411705 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e8058-a60b-4da8-89cf-fc77dc76b387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:54.412638 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e8058-a60b-4da8-89cf-fc77dc76b387 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:54.413109 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e8058-a60b-4da8-89cf-fc77dc76b387 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:54.413397 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77e8058-a60b-4da8-89cf-fc77dc76b387 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:54.426943 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949254b3-bd56-4712-89d7-631fd8897a1d None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:54.465780 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7724b-ec98-492c-ab5d-e5cdc976d24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:54.467808 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7724b-ec98-492c-ab5d-e5cdc976d24e None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:54.467808 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7724b-ec98-492c-ab5d-e5cdc976d24e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:54.467970 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d7724b-ec98-492c-ab5d-e5cdc976d24e None admin] PATH_INFO: `/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:54.469431 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d7724b-ec98-492c-ab5d-e5cdc976d24e None admin] RBAC: Authorizing `identity:get_project(project_id=49b1ed5ea8fc417f8247e689e804e6f1)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:54.470989 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d7724b-ec98-492c-ab5d-e5cdc976d24e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:54.473603 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 39/87] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:53:54 2022] GET /identity/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:53:54.474549 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7724b-ec98-492c-ab5d-e5cdc976d24e None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:54.480501 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:54.488930 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77e8058-a60b-4da8-89cf-fc77dc76b387 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:54.490491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77e8058-a60b-4da8-89cf-fc77dc76b387 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:54.598856 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d7724b-ec98-492c-ab5d-e5cdc976d24e None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:54.613982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 49/88] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:53:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 16:53:54.615211 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e8058-a60b-4da8-89cf-fc77dc76b387 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:54.619051 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:54.644414 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb1ccfc-a59c-4a75-b2fb-d33a6a6803e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:54.645013 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1ccfc-a59c-4a75-b2fb-d33a6a6803e2 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:54.645302 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1ccfc-a59c-4a75-b2fb-d33a6a6803e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:54.645744 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb1ccfc-a59c-4a75-b2fb-d33a6a6803e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:54.646600 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb1ccfc-a59c-4a75-b2fb-d33a6a6803e2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:53:54.646903 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb1ccfc-a59c-4a75-b2fb-d33a6a6803e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:54.649112 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb1ccfc-a59c-4a75-b2fb-d33a6a6803e2 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:54.661077 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 40/89] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 6 16:53:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:53:54.661965 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb1ccfc-a59c-4a75-b2fb-d33a6a6803e2 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:54.734216 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:54.757033 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77e8058-a60b-4da8-89cf-fc77dc76b387 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:54.770440 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb1ccfc-a59c-4a75-b2fb-d33a6a6803e2 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:54.805498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527f166-49da-4b2f-b3e7-7d7e99fd7b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:54.806700 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527f166-49da-4b2f-b3e7-7d7e99fd7b40 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:54.807192 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527f166-49da-4b2f-b3e7-7d7e99fd7b40 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:54.807779 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c527f166-49da-4b2f-b3e7-7d7e99fd7b40 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:54.808479 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c1d406-1ec7-4952-8161-4753132de6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:54.809157 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c1d406-1ec7-4952-8161-4753132de6e5 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:54.809261 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527f166-49da-4b2f-b3e7-7d7e99fd7b40 None admin] RBAC: Adding query filter params (role.id=7fd63d2d99254c00bb3501173643ee30, scope.project.id=49b1ed5ea8fc417f8247e689e804e6f1, user.id=8ad29aae43d24936a5dbec7f54883329) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:53:54.809447 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c1d406-1ec7-4952-8161-4753132de6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:54.809734 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c1d406-1ec7-4952-8161-4753132de6e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:54.810170 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527f166-49da-4b2f-b3e7-7d7e99fd7b40 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:54.810581 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c1d406-1ec7-4952-8161-4753132de6e5 None admin] RBAC: Adding query filter params (role.id=b41f8a7a58e64f749f2099ca45cc04a7, scope.project.id=49b1ed5ea8fc417f8247e689e804e6f1, user.id=fb49b9dd86cd415bb41412ca15c4d0ba) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:53:54.810901 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c1d406-1ec7-4952-8161-4753132de6e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:54.812274 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c527f166-49da-4b2f-b3e7-7d7e99fd7b40 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:54.812379 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c1d406-1ec7-4952-8161-4753132de6e5 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:54.819636 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 41/90] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:53:54 2022] GET /identity/v3/role_assignments?user.id=fb49b9dd86cd415bb41412ca15c4d0ba&scope.project.id=49b1ed5ea8fc417f8247e689e804e6f1&role.id=b41f8a7a58e64f749f2099ca45cc04a7 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:53:54.820305 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c1d406-1ec7-4952-8161-4753132de6e5 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:54.822295 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 50/91] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:53:54 2022] GET /identity/v3/role_assignments?user.id=8ad29aae43d24936a5dbec7f54883329&scope.project.id=49b1ed5ea8fc417f8247e689e804e6f1&role.id=7fd63d2d99254c00bb3501173643ee30 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:53:54.823147 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527f166-49da-4b2f-b3e7-7d7e99fd7b40 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:54.939985 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:54.976239 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c1d406-1ec7-4952-8161-4753132de6e5 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:55.014753 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:55.029247 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85958a2-7eea-4a96-a0db-ea034dd231ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:55.030043 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85958a2-7eea-4a96-a0db-ea034dd231ff None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:55.030337 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85958a2-7eea-4a96-a0db-ea034dd231ff None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:55.030620 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85958a2-7eea-4a96-a0db-ea034dd231ff None admin] PATH_INFO: `/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:55.032950 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85958a2-7eea-4a96-a0db-ea034dd231ff None admin] RBAC: Authorizing `identity:get_project(project_id=49b1ed5ea8fc417f8247e689e804e6f1)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:55.034438 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85958a2-7eea-4a96-a0db-ea034dd231ff None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:55.036603 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 42/92] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:53:54 2022] GET /identity/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1 => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:53:55.041839 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c527f166-49da-4b2f-b3e7-7d7e99fd7b40 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:55.042598 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85958a2-7eea-4a96-a0db-ea034dd231ff None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:55.089297 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f4a64d-09ff-41eb-a1a6-a270ef0e72fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:55.090244 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4a64d-09ff-41eb-a1a6-a270ef0e72fc None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:55.090530 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4a64d-09ff-41eb-a1a6-a270ef0e72fc None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:55.090811 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f4a64d-09ff-41eb-a1a6-a270ef0e72fc None admin] PATH_INFO: `/v3/users/8ad29aae43d24936a5dbec7f54883329` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:55.092080 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f4a64d-09ff-41eb-a1a6-a270ef0e72fc None admin] RBAC: Authorizing `identity:get_user(user_id=8ad29aae43d24936a5dbec7f54883329)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:55.093536 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f4a64d-09ff-41eb-a1a6-a270ef0e72fc None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:55.095497 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 51/93] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:53:54 2022] GET /identity/v3/users/8ad29aae43d24936a5dbec7f54883329 => generated 275 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:53:55.100716 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f4a64d-09ff-41eb-a1a6-a270ef0e72fc None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:55.200337 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85958a2-7eea-4a96-a0db-ea034dd231ff None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:55.234846 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e5ff4e-a071-4231-8bba-0879b8d71109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:55.235513 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5ff4e-a071-4231-8bba-0879b8d71109 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:55.235829 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5ff4e-a071-4231-8bba-0879b8d71109 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:55.236112 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5ff4e-a071-4231-8bba-0879b8d71109 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:55.236950 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e5ff4e-a071-4231-8bba-0879b8d71109 None admin] RBAC: Adding query filter params (role.id=c31c5ee7ed8646e48fa81e85d10fcaf2, scope.project.id=49b1ed5ea8fc417f8247e689e804e6f1, user.id=8ad29aae43d24936a5dbec7f54883329) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:53:55.237265 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e5ff4e-a071-4231-8bba-0879b8d71109 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:55.238726 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e5ff4e-a071-4231-8bba-0879b8d71109 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:55.244445 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 43/94] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:53:55 2022] GET /identity/v3/role_assignments?user.id=8ad29aae43d24936a5dbec7f54883329&scope.project.id=49b1ed5ea8fc417f8247e689e804e6f1&role.id=c31c5ee7ed8646e48fa81e85d10fcaf2 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:53:55.317949 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f4a64d-09ff-41eb-a1a6-a270ef0e72fc None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:55.432473 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a42f7c0-cee2-4930-be8e-23be5fb81446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:55.433197 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42f7c0-cee2-4930-be8e-23be5fb81446 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:55.433497 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42f7c0-cee2-4930-be8e-23be5fb81446 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:55.438458 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a42f7c0-cee2-4930-be8e-23be5fb81446 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:55.440824 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:55.563346 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a42f7c0-cee2-4930-be8e-23be5fb81446 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:55.564789 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a42f7c0-cee2-4930-be8e-23be5fb81446 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:55.765963 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 52/95] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:53:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 666 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 16:53:55.772388 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e5ff4e-a071-4231-8bba-0879b8d71109 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:55.884679 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:55.944062 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e5ff4e-a071-4231-8bba-0879b8d71109 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:56.005827 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd83584-09c4-4d7c-931c-7266d732ec1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:56.007069 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd83584-09c4-4d7c-931c-7266d732ec1f None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:56.007561 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd83584-09c4-4d7c-931c-7266d732ec1f None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:56.008063 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd83584-09c4-4d7c-931c-7266d732ec1f None admin] PATH_INFO: `/v3/projects/a1142bf71f5b4e6e8c84cd2a9330b15d` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:56.012587 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd83584-09c4-4d7c-931c-7266d732ec1f None admin] RBAC: Authorizing `identity:get_project(project_id=a1142bf71f5b4e6e8c84cd2a9330b15d)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:56.014307 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd83584-09c4-4d7c-931c-7266d732ec1f None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:56.016607 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 44/96] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:53:55 2022] GET /identity/v3/projects/a1142bf71f5b4e6e8c84cd2a9330b15d => generated 312 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:53:56.022362 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a42f7c0-cee2-4930-be8e-23be5fb81446 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:56.091760 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:56.191507 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a42f7c0-cee2-4930-be8e-23be5fb81446 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:56.240433 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43034a-6ec5-480f-9867-65066736c8ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:56.241117 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43034a-6ec5-480f-9867-65066736c8ac None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:56.241412 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43034a-6ec5-480f-9867-65066736c8ac None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:56.241698 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43034a-6ec5-480f-9867-65066736c8ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:56.242546 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf43034a-6ec5-480f-9867-65066736c8ac None admin] RBAC: Adding query filter params (role.id=7fd63d2d99254c00bb3501173643ee30, scope.project.id=a1142bf71f5b4e6e8c84cd2a9330b15d, user.id=8ad29aae43d24936a5dbec7f54883329) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:53:56.242864 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf43034a-6ec5-480f-9867-65066736c8ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:56.244352 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf43034a-6ec5-480f-9867-65066736c8ac None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:56.250873 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 53/97] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:53:56 2022] GET /identity/v3/role_assignments?user.id=8ad29aae43d24936a5dbec7f54883329&scope.project.id=a1142bf71f5b4e6e8c84cd2a9330b15d&role.id=7fd63d2d99254c00bb3501173643ee30 => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:53:57.079950 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011e83d-a176-40f3-8a00-8c9bea9c2fc4 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:57.080960 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011e83d-a176-40f3-8a00-8c9bea9c2fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:57.081453 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1011e83d-a176-40f3-8a00-8c9bea9c2fc4 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:57.083019 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 45/98] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:53:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:53:57.088369 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8309c-3fad-4278-903f-977c92602a1e None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:57.089197 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8309c-3fad-4278-903f-977c92602a1e None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:57.090296 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a8309c-3fad-4278-903f-977c92602a1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:57.113646 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-30a8309c-3fad-4278-903f-977c92602a1e None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:53:57.115146 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-30a8309c-3fad-4278-903f-977c92602a1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:53:57.188491 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7bc93-9b0e-4668-8f73-baad788c4296 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:57.188825 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7bc93-9b0e-4668-8f73-baad788c4296 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:57.189111 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7bc93-9b0e-4668-8f73-baad788c4296 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:57.190737 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 46/99] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:53:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:53:57.195324 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889b422-b4d2-4a76-95ab-9a9688a55c64 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:57.195617 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889b422-b4d2-4a76-95ab-9a9688a55c64 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:57.195929 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889b422-b4d2-4a76-95ab-9a9688a55c64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:57.220491 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-3889b422-b4d2-4a76-95ab-9a9688a55c64 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:53:57.221693 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-3889b422-b4d2-4a76-95ab-9a9688a55c64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:53:57.228183 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 54/100] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:53:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:53:57.258773 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a8309c-3fad-4278-903f-977c92602a1e None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:57.313333 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:57.330727 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 47/101] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:53:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 16:53:57.362392 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3889b422-b4d2-4a76-95ab-9a9688a55c64 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:57.448078 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:57.461958 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a8309c-3fad-4278-903f-977c92602a1e None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:57.488128 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3889b422-b4d2-4a76-95ab-9a9688a55c64 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:57.534725 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb870f-59e8-4733-8040-54916fc1c8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:57.535938 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb870f-59e8-4733-8040-54916fc1c8a2 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:57.536534 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb870f-59e8-4733-8040-54916fc1c8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:57.537011 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb870f-59e8-4733-8040-54916fc1c8a2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:57.544645 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb870f-59e8-4733-8040-54916fc1c8a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:57.546661 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb870f-59e8-4733-8040-54916fc1c8a2 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:57.548878 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 55/102] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:53:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:53:57.550695 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd6be13-3fcd-4926-94af-2b07cb2c993b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:57.551531 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6be13-3fcd-4926-94af-2b07cb2c993b None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:57.551845 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6be13-3fcd-4926-94af-2b07cb2c993b None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:57.552124 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd6be13-3fcd-4926-94af-2b07cb2c993b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:57.555079 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb870f-59e8-4733-8040-54916fc1c8a2 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:57.569200 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd6be13-3fcd-4926-94af-2b07cb2c993b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:53:57.586316 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-2bd6be13-3fcd-4926-94af-2b07cb2c993b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 16:53:57.586316 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:53:57.586316 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:53:57.586316 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:53:57.586316 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:53:57.586316 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:53:57.586316 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:53:57.586316 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:53:57.586316 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:53:57.586316 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:53:57.586316 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:53:57.586316 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:53:57.586316 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:53:57.586316 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:53:57.586316 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:53:57.586316 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:53:57.587207 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:53:57.587207 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:53:57.587207 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:53:57.587207 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:53:57.587207 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:53:57.587207 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:53:57.587207 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:53:57.587207 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:53:57.587207 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:53:57.587207 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:53:57.587207 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:53:57.587207 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:53:57.587207 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:53:57.587207 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:53:57.587207 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:53:57.587207 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:53:57.588404 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:53:57.588404 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:53:57.588404 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:53:57.588404 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:53:57.588404 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:53:57.588404 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:53:57.588404 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:53:57.588404 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:53:57.588404 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:53:57.588404 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:53:57.588404 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 16:53:57.588404 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:53:57.588404 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 48/103] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:53:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:53:57.593239 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd6be13-3fcd-4926-94af-2b07cb2c993b None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:57.743348 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd6be13-3fcd-4926-94af-2b07cb2c993b None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:57.790754 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:57.795413 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:57.798271 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb870f-59e8-4733-8040-54916fc1c8a2 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:57.833400 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252e23b-d3fa-4bcb-85b7-4c922f1ce61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:57.834379 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252e23b-d3fa-4bcb-85b7-4c922f1ce61b None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:57.834840 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252e23b-d3fa-4bcb-85b7-4c922f1ce61b None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:57.835292 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8252e23b-d3fa-4bcb-85b7-4c922f1ce61b None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:57.836286 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252e23b-d3fa-4bcb-85b7-4c922f1ce61b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:53:57.836823 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252e23b-d3fa-4bcb-85b7-4c922f1ce61b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:57.838428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8252e23b-d3fa-4bcb-85b7-4c922f1ce61b None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:57.856000 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 49/104] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 6 16:53:57 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:53:57.861772 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252e23b-d3fa-4bcb-85b7-4c922f1ce61b None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:57.900635 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad318d3-f271-40fa-9f59-f512c08eb181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:57.901545 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad318d3-f271-40fa-9f59-f512c08eb181 None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:57.901865 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad318d3-f271-40fa-9f59-f512c08eb181 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:57.902145 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad318d3-f271-40fa-9f59-f512c08eb181 None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:57.902884 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad318d3-f271-40fa-9f59-f512c08eb181 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:57.904798 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad318d3-f271-40fa-9f59-f512c08eb181 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:58.005504 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8252e23b-d3fa-4bcb-85b7-4c922f1ce61b None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:58.033342 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 56/105] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Jul 6 16:53:57 2022] POST /identity/v3/projects => generated 302 bytes in 479 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 16:53:58.035168 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58052688-dff5-45f7-9abb-e534e1dec497 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:58.035611 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58052688-dff5-45f7-9abb-e534e1dec497 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:58.036092 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58052688-dff5-45f7-9abb-e534e1dec497 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:58.037585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 57/106] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:53:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:53:58.046636 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283cdc1-8dc0-456e-b7d3-687b6bb8420c None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:58.047265 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283cdc1-8dc0-456e-b7d3-687b6bb8420c None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:58.047957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d283cdc1-8dc0-456e-b7d3-687b6bb8420c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:58.051766 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2616bb9-6e31-4b53-82fa-418484c1f904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:58.051766 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2616bb9-6e31-4b53-82fa-418484c1f904 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:58.051766 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2616bb9-6e31-4b53-82fa-418484c1f904 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:58.051766 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2616bb9-6e31-4b53-82fa-418484c1f904 None admin] PATH_INFO: `/v3/users/025c05bc4a8341ddb109caab361c5d41` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:58.068815 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2616bb9-6e31-4b53-82fa-418484c1f904 None admin] RBAC: Authorizing `identity:get_user(user_id=025c05bc4a8341ddb109caab361c5d41)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:58.070672 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2616bb9-6e31-4b53-82fa-418484c1f904 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:58.073131 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 50/107] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:53:57 2022] GET /identity/v3/users/025c05bc4a8341ddb109caab361c5d41 => generated 289 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:53:58.078328 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2616bb9-6e31-4b53-82fa-418484c1f904 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:58.080513 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-d283cdc1-8dc0-456e-b7d3-687b6bb8420c None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:53:58.081721 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d283cdc1-8dc0-456e-b7d3-687b6bb8420c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:53:58.148201 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:58.229826 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 58/108] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:53:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:53:58.232707 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb4f840-a6bf-4fb8-86ae-cd05d66bced5 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:58.233205 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb4f840-a6bf-4fb8-86ae-cd05d66bced5 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:58.233683 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb4f840-a6bf-4fb8-86ae-cd05d66bced5 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:58.241162 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 59/109] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:53:58 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:53:58.247322 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7eef78-3e13-41b0-9a02-1a903dd7a511 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:58.247868 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7eef78-3e13-41b0-9a02-1a903dd7a511 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:58.248228 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2616bb9-6e31-4b53-82fa-418484c1f904 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:58.248546 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7eef78-3e13-41b0-9a02-1a903dd7a511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:58.315533 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-8c7eef78-3e13-41b0-9a02-1a903dd7a511 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:53:58.318383 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-8c7eef78-3e13-41b0-9a02-1a903dd7a511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:53:58.340735 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d0020b-d3af-4457-a508-b1d9bd2c9b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:58.341449 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d0020b-d3af-4457-a508-b1d9bd2c9b3e None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:58.341734 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d0020b-d3af-4457-a508-b1d9bd2c9b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:58.342052 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d0020b-d3af-4457-a508-b1d9bd2c9b3e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:58.398873 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 60/110] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:53:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 16:53:58.400245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7eef78-3e13-41b0-9a02-1a903dd7a511 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:58.413147 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:58.420600 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d0020b-d3af-4457-a508-b1d9bd2c9b3e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:58.422295 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d0020b-d3af-4457-a508-b1d9bd2c9b3e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:58.466228 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:58.512412 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7eef78-3e13-41b0-9a02-1a903dd7a511 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:58.532676 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 51/111] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:53:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 16:53:58.533621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d0020b-d3af-4457-a508-b1d9bd2c9b3e None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:58.564090 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427bffa5-1e2a-42dd-a965-fd605c61cc5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:58.565459 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427bffa5-1e2a-42dd-a965-fd605c61cc5c None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:58.565978 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427bffa5-1e2a-42dd-a965-fd605c61cc5c None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:58.566461 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427bffa5-1e2a-42dd-a965-fd605c61cc5c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:58.578195 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427bffa5-1e2a-42dd-a965-fd605c61cc5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:53:58.587035 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-427bffa5-1e2a-42dd-a965-fd605c61cc5c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:53:58.587035 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:53:58.587035 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:53:58.587035 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:53:58.587035 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:53:58.587035 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:53:58.587035 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:53:58.587035 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:53:58.587035 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:53:58.587035 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:53:58.587035 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:53:58.587035 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:53:58.587035 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:53:58.587035 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:53:58.587035 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:53:58.587035 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:53:58.587934 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:53:58.587934 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:53:58.587934 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:53:58.587934 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:53:58.587934 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:53:58.587934 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:53:58.587934 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:53:58.587934 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:53:58.587934 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:53:58.587934 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:53:58.587934 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:53:58.587934 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:53:58.587934 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:53:58.587934 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:53:58.587934 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:53:58.587934 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:53:58.588694 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:53:58.588694 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:53:58.588694 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:53:58.588694 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:53:58.588694 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:53:58.588694 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:53:58.588694 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:53:58.588694 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:53:58.588694 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:53:58.588694 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:53:58.588694 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:53:58.588694 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:53:58.596147 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 61/112] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 6 16:53:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:53:58.597262 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427bffa5-1e2a-42dd-a965-fd605c61cc5c None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:58.641432 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d0020b-d3af-4457-a508-b1d9bd2c9b3e None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:58.674450 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:58.679517 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6878c1-478e-48ea-8e3d-5db61eb6e0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:58.680440 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6878c1-478e-48ea-8e3d-5db61eb6e0a1 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:58.680721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6878c1-478e-48ea-8e3d-5db61eb6e0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:58.680992 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6878c1-478e-48ea-8e3d-5db61eb6e0a1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:58.689400 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:58.692494 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6878c1-478e-48ea-8e3d-5db61eb6e0a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:53:58.700881 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-bb6878c1-478e-48ea-8e3d-5db61eb6e0a1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 16:53:58.700881 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:53:58.700881 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:53:58.700881 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:53:58.700881 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:53:58.700881 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:53:58.700881 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:53:58.700881 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:53:58.700881 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:53:58.700881 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:53:58.700881 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:53:58.700881 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:53:58.700881 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:53:58.700881 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:53:58.700881 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:53:58.700881 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:53:58.701843 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:53:58.701843 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:53:58.701843 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:53:58.701843 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:53:58.701843 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:53:58.701843 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:53:58.701843 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:53:58.701843 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:53:58.701843 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:53:58.701843 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:53:58.701843 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:53:58.701843 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:53:58.701843 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:53:58.701843 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:53:58.701843 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:53:58.701843 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:53:58.702981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:53:58.702981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:53:58.702981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:53:58.702981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:53:58.702981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:53:58.702981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:53:58.702981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:53:58.702981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:53:58.702981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:53:58.702981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:53:58.702981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 16:53:58.702981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:53:58.702981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 52/113] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:53:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:53:58.702981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6878c1-478e-48ea-8e3d-5db61eb6e0a1 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:58.714119 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427bffa5-1e2a-42dd-a965-fd605c61cc5c None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:58.752642 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ced62-1bc0-4a3a-8202-733b7ec482f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:58.754240 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ced62-1bc0-4a3a-8202-733b7ec482f2 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:58.754747 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ced62-1bc0-4a3a-8202-733b7ec482f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:58.755266 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ced62-1bc0-4a3a-8202-733b7ec482f2 None admin] PATH_INFO: `/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:58.756754 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8ced62-1bc0-4a3a-8202-733b7ec482f2 None admin] RBAC: Authorizing `identity:get_project(project_id=49b1ed5ea8fc417f8247e689e804e6f1)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:58.758575 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8ced62-1bc0-4a3a-8202-733b7ec482f2 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:58.760583 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 62/114] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:53:58 2022] GET /identity/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:53:58.762205 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ced62-1bc0-4a3a-8202-733b7ec482f2 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:58.817963 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6878c1-478e-48ea-8e3d-5db61eb6e0a1 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:58.854053 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e89c1f-08f3-4407-b8d7-b8c8e8b8c524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:58.855400 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e89c1f-08f3-4407-b8d7-b8c8e8b8c524 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:58.855632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e89c1f-08f3-4407-b8d7-b8c8e8b8c524 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:58.856008 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e89c1f-08f3-4407-b8d7-b8c8e8b8c524 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:58.856993 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e89c1f-08f3-4407-b8d7-b8c8e8b8c524 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:53:58.857166 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e89c1f-08f3-4407-b8d7-b8c8e8b8c524 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:58.858556 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e89c1f-08f3-4407-b8d7-b8c8e8b8c524 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:58.868015 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 53/115] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:53:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:53:58.869444 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36b445-ba61-4b09-8493-c4a732251ec6 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:58.869850 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36b445-ba61-4b09-8493-c4a732251ec6 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:58.870134 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b36b445-ba61-4b09-8493-c4a732251ec6 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:58.872470 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 54/116] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:53:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:53:58.873143 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b36b445-ba61-4b09-8493-c4a732251ec6 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:58.876513 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8ced62-1bc0-4a3a-8202-733b7ec482f2 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:58.914268 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a39b2f-053e-4149-a411-e7f3794a66a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:58.915410 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a39b2f-053e-4149-a411-e7f3794a66a2 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:58.916036 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a39b2f-053e-4149-a411-e7f3794a66a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:58.916517 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a39b2f-053e-4149-a411-e7f3794a66a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:58.918177 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a39b2f-053e-4149-a411-e7f3794a66a2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:53:58.918687 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a39b2f-053e-4149-a411-e7f3794a66a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:58.920387 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a39b2f-053e-4149-a411-e7f3794a66a2 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:58.926683 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:58.931292 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 63/117] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 6 16:53:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:53:58.932559 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a39b2f-053e-4149-a411-e7f3794a66a2 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:58.960117 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:58.999898 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b36b445-ba61-4b09-8493-c4a732251ec6 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:59.041214 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b2ac0-c4d7-4fd0-aee6-805cb3011ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:59.041884 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b2ac0-c4d7-4fd0-aee6-805cb3011ffd None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:59.042173 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b2ac0-c4d7-4fd0-aee6-805cb3011ffd None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:59.042996 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b2ac0-c4d7-4fd0-aee6-805cb3011ffd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:59.043861 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873b2ac0-c4d7-4fd0-aee6-805cb3011ffd None admin] RBAC: Adding query filter params (role.id=e9f69ea47fce47fab94435de5a29de5f, scope.project.id=49b1ed5ea8fc417f8247e689e804e6f1, user.id=025c05bc4a8341ddb109caab361c5d41) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:53:59.044184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873b2ac0-c4d7-4fd0-aee6-805cb3011ffd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:59.045673 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873b2ac0-c4d7-4fd0-aee6-805cb3011ffd None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:59.051595 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 55/118] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:53:58 2022] GET /identity/v3/role_assignments?user.id=025c05bc4a8341ddb109caab361c5d41&scope.project.id=49b1ed5ea8fc417f8247e689e804e6f1&role.id=e9f69ea47fce47fab94435de5a29de5f => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:53:59.053007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0b277-e18d-423a-ae2d-6325d2dc8a61 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:59.053306 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0b277-e18d-423a-ae2d-6325d2dc8a61 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:59.053592 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d0b277-e18d-423a-ae2d-6325d2dc8a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:59.063766 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a39b2f-053e-4149-a411-e7f3794a66a2 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:59.071215 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-e1d0b277-e18d-423a-ae2d-6325d2dc8a61 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:53:59.072488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e1d0b277-e18d-423a-ae2d-6325d2dc8a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:53:59.104715 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a79f5eb-9395-40fc-8697-5f00d0cee2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:59.106265 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79f5eb-9395-40fc-8697-5f00d0cee2e3 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:59.106557 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79f5eb-9395-40fc-8697-5f00d0cee2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:59.106833 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a79f5eb-9395-40fc-8697-5f00d0cee2e3 None admin] PATH_INFO: `/v3/users/fb49b9dd86cd415bb41412ca15c4d0ba` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:59.108176 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a79f5eb-9395-40fc-8697-5f00d0cee2e3 None admin] RBAC: Authorizing `identity:get_user(user_id=fb49b9dd86cd415bb41412ca15c4d0ba)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:59.109709 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a79f5eb-9395-40fc-8697-5f00d0cee2e3 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:59.111948 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 64/119] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:53:58 2022] GET /identity/v3/users/fb49b9dd86cd415bb41412ca15c4d0ba => generated 247 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:53:59.112685 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a79f5eb-9395-40fc-8697-5f00d0cee2e3 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:59.146439 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 56/120] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:53:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:53:59.147223 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d0b277-e18d-423a-ae2d-6325d2dc8a61 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:59.225881 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:59.226473 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:59.264793 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a79f5eb-9395-40fc-8697-5f00d0cee2e3 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:59.308540 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48137e04-ed28-46a1-899f-8c8050259d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:59.309520 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48137e04-ed28-46a1-899f-8c8050259d04 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:59.309809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48137e04-ed28-46a1-899f-8c8050259d04 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:59.310086 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48137e04-ed28-46a1-899f-8c8050259d04 None admin] PATH_INFO: `/v3/users/8ad29aae43d24936a5dbec7f54883329` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:59.311309 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48137e04-ed28-46a1-899f-8c8050259d04 None admin] RBAC: Authorizing `identity:get_user(user_id=8ad29aae43d24936a5dbec7f54883329)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:59.312822 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48137e04-ed28-46a1-899f-8c8050259d04 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:59.314796 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 65/121] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:53:59 2022] GET /identity/v3/users/8ad29aae43d24936a5dbec7f54883329 => generated 275 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:53:59.315225 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48137e04-ed28-46a1-899f-8c8050259d04 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:59.340105 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d0b277-e18d-423a-ae2d-6325d2dc8a61 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:59.421912 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48137e04-ed28-46a1-899f-8c8050259d04 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:59.448522 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a7b0aa-a68a-46d4-b63c-db6e07dcab8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:59.449943 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7b0aa-a68a-46d4-b63c-db6e07dcab8b None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:59.450242 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7b0aa-a68a-46d4-b63c-db6e07dcab8b None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:59.450519 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a7b0aa-a68a-46d4-b63c-db6e07dcab8b None admin] PATH_INFO: `/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:59.454761 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a7b0aa-a68a-46d4-b63c-db6e07dcab8b None admin] RBAC: Authorizing `identity:get_project(project_id=49b1ed5ea8fc417f8247e689e804e6f1)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:59.456501 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a7b0aa-a68a-46d4-b63c-db6e07dcab8b None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:59.456889 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca97a85-a70d-4530-801f-014759714945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:59.460946 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca97a85-a70d-4530-801f-014759714945 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:59.460946 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca97a85-a70d-4530-801f-014759714945 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:59.461121 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 57/122] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:53:59 2022] GET /identity/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1 => generated 298 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:53:59.461194 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca97a85-a70d-4530-801f-014759714945 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:59.461259 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a7b0aa-a68a-46d4-b63c-db6e07dcab8b None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:59.467277 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:59.473365 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca97a85-a70d-4530-801f-014759714945 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:53:59.483301 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-eca97a85-a70d-4530-801f-014759714945 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 16:53:59.483301 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:53:59.483301 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:53:59.483301 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:53:59.483301 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:53:59.483301 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:53:59.483301 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:53:59.483301 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:53:59.483301 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:53:59.483301 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:53:59.483301 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:53:59.483301 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:53:59.483301 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:53:59.483301 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:53:59.483301 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:53:59.483301 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:53:59.492040 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:53:59.492040 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:53:59.492040 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:53:59.492040 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:53:59.492040 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:53:59.492040 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:53:59.492040 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:53:59.492040 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:53:59.492040 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:53:59.492040 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:53:59.492040 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:53:59.492040 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:53:59.492040 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:53:59.492040 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:53:59.492040 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:53:59.492040 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:53:59.492814 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:53:59.492814 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:53:59.492814 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:53:59.492814 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:53:59.492814 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:53:59.492814 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:53:59.492814 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:53:59.492814 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:53:59.492814 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:53:59.492814 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:53:59.492814 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 16:53:59.492814 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:53:59.492814 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 66/123] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 6 16:53:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:53:59.492814 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca97a85-a70d-4530-801f-014759714945 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:59.602323 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a7b0aa-a68a-46d4-b63c-db6e07dcab8b None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:59.602676 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca97a85-a70d-4530-801f-014759714945 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:59.626990 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:59.637663 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14492dac-6b78-4088-b9d9-67df0e54abd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:59.638615 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14492dac-6b78-4088-b9d9-67df0e54abd4 None admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:59.638888 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14492dac-6b78-4088-b9d9-67df0e54abd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:59.639174 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14492dac-6b78-4088-b9d9-67df0e54abd4 None admin] PATH_INFO: `/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1/users/fb49b9dd86cd415bb41412ca15c4d0ba/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:59.645493 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32e88f-905c-4913-8af6-9d2ff805b19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:59.645657 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14492dac-6b78-4088-b9d9-67df0e54abd4 None admin] RBAC: Authorizing `identity:create_grant(project_id=49b1ed5ea8fc417f8247e689e804e6f1, user_id=fb49b9dd86cd415bb41412ca15c4d0ba, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:59.645748 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32e88f-905c-4913-8af6-9d2ff805b19c None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:59.645748 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32e88f-905c-4913-8af6-9d2ff805b19c None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:59.645748 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e32e88f-905c-4913-8af6-9d2ff805b19c None admin] PATH_INFO: `/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:59.645995 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14492dac-6b78-4088-b9d9-67df0e54abd4 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:59.646343 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e32e88f-905c-4913-8af6-9d2ff805b19c None admin] RBAC: Authorizing `identity:get_project(project_id=49b1ed5ea8fc417f8247e689e804e6f1)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:59.646343 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e32e88f-905c-4913-8af6-9d2ff805b19c None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:59.648081 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 58/124] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:53:59 2022] GET /identity/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:53:59.649280 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32e88f-905c-4913-8af6-9d2ff805b19c None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:59.726975 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 67/125] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 6 16:53:59 2022] PUT /identity/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1/users/fb49b9dd86cd415bb41412ca15c4d0ba/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 16:53:59.735016 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77820eb2-414a-4bf2-aa7a-3f6c4926fb2a None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:59.735717 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77820eb2-414a-4bf2-aa7a-3f6c4926fb2a None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:59.736265 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77820eb2-414a-4bf2-aa7a-3f6c4926fb2a None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:59.738136 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 68/126] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:53:59 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:53:59.739250 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77820eb2-414a-4bf2-aa7a-3f6c4926fb2a None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:59.763776 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e32e88f-905c-4913-8af6-9d2ff805b19c None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:59.798769 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c54e3e-2477-4a29-9b22-52d17460ec75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:59.799285 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c54e3e-2477-4a29-9b22-52d17460ec75 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:59.799574 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c54e3e-2477-4a29-9b22-52d17460ec75 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:59.799909 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c54e3e-2477-4a29-9b22-52d17460ec75 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:59.800672 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c54e3e-2477-4a29-9b22-52d17460ec75 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:53:59.800984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c54e3e-2477-4a29-9b22-52d17460ec75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:59.802412 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c54e3e-2477-4a29-9b22-52d17460ec75 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:53:59.812436 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 59/127] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:53:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:53:59.813509 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc981f-c633-4d7e-9f9e-aa1f6c60c3ca None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:59.814367 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc981f-c633-4d7e-9f9e-aa1f6c60c3ca None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:59.814367 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bc981f-c633-4d7e-9f9e-aa1f6c60c3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:59.832991 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-17bc981f-c633-4d7e-9f9e-aa1f6c60c3ca None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:53:59.834472 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-17bc981f-c633-4d7e-9f9e-aa1f6c60c3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:53:59.876106 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:59.892419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:53:59.920371 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 60/128] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:53:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:53:59.920371 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc981f-c633-4d7e-9f9e-aa1f6c60c3ca None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:53:59.954779 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77820eb2-414a-4bf2-aa7a-3f6c4926fb2a None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:53:59.993764 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b1a5a4-965a-4e94-a20a-0455b99df57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:53:59.994754 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1a5a4-965a-4e94-a20a-0455b99df57e None admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:53:59.995161 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1a5a4-965a-4e94-a20a-0455b99df57e None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:53:59.995578 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1a5a4-965a-4e94-a20a-0455b99df57e None admin] PATH_INFO: `/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1/users/8ad29aae43d24936a5dbec7f54883329/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:53:59.997978 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b1a5a4-965a-4e94-a20a-0455b99df57e None admin] RBAC: Authorizing `identity:create_grant(project_id=49b1ed5ea8fc417f8247e689e804e6f1, user_id=8ad29aae43d24936a5dbec7f54883329, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:53:59.999623 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b1a5a4-965a-4e94-a20a-0455b99df57e None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:00.027208 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bc981f-c633-4d7e-9f9e-aa1f6c60c3ca None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:00.059440 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 69/129] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 6 16:53:59 2022] PUT /identity/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1/users/8ad29aae43d24936a5dbec7f54883329/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 16:54:00.060962 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b1a5a4-965a-4e94-a20a-0455b99df57e None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:00.092806 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6ea97-7874-43bf-a0cb-8bdc8c9e578c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:00.095077 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6ea97-7874-43bf-a0cb-8bdc8c9e578c None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:00.095810 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6ea97-7874-43bf-a0cb-8bdc8c9e578c None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:00.097797 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6ea97-7874-43bf-a0cb-8bdc8c9e578c None admin] PATH_INFO: `/v3/users/8ad29aae43d24936a5dbec7f54883329` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:00.100469 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c6ea97-7874-43bf-a0cb-8bdc8c9e578c None admin] RBAC: Authorizing `identity:get_user(user_id=8ad29aae43d24936a5dbec7f54883329)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:00.105550 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c6ea97-7874-43bf-a0cb-8bdc8c9e578c None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:00.108253 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 61/130] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:53:59 2022] GET /identity/v3/users/8ad29aae43d24936a5dbec7f54883329 => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:00.114290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6ea97-7874-43bf-a0cb-8bdc8c9e578c None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:00.225023 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:00.255137 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b1a5a4-965a-4e94-a20a-0455b99df57e None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:00.276319 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:00.300800 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3bdfcb-6b31-432e-b995-2bd1bfc906fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:00.301673 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3bdfcb-6b31-432e-b995-2bd1bfc906fe None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:00.301971 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3bdfcb-6b31-432e-b995-2bd1bfc906fe None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:00.302274 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3bdfcb-6b31-432e-b995-2bd1bfc906fe None admin] PATH_INFO: `/v3/roles/member` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:00.315949 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3bdfcb-6b31-432e-b995-2bd1bfc906fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:00.325549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-be3bdfcb-6b31-432e-b995-2bd1bfc906fe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 16:54:00.325549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:00.325549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:00.325549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:00.325549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:00.325549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:00.325549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:00.325549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:00.325549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:00.325549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:00.325549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:00.325549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:00.325549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:00.325549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:00.325549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:00.325549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:00.326677 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:00.326677 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:00.326677 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:00.326677 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:00.326677 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:00.326677 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:00.326677 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:00.326677 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:00.326677 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:00.326677 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:00.326677 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:00.326677 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:00.326677 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:00.326677 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:00.326677 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:00.326677 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:00.327430 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:00.327430 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:00.327430 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:00.327430 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:00.327430 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:00.327430 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:00.327430 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:00.327430 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:00.327430 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:00.327430 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:00.327430 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 16:54:00.327430 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:00.327430 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 70/131] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:54:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:00.336090 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3bdfcb-6b31-432e-b995-2bd1bfc906fe None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:00.382047 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6ea97-7874-43bf-a0cb-8bdc8c9e578c None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:00.426975 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc62b7c-67e9-4d81-b801-36ecc1691205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:00.427906 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc62b7c-67e9-4d81-b801-36ecc1691205 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:00.428196 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc62b7c-67e9-4d81-b801-36ecc1691205 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:00.429540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc62b7c-67e9-4d81-b801-36ecc1691205 None admin] PATH_INFO: `/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:00.432525 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc62b7c-67e9-4d81-b801-36ecc1691205 None admin] RBAC: Authorizing `identity:get_project(project_id=49b1ed5ea8fc417f8247e689e804e6f1)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:00.434590 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc62b7c-67e9-4d81-b801-36ecc1691205 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:00.437844 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 62/132] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:54:00 2022] GET /identity/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1 => generated 298 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:00.447293 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc62b7c-67e9-4d81-b801-36ecc1691205 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:00.530179 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3bdfcb-6b31-432e-b995-2bd1bfc906fe None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:00.604931 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90833c7-efd4-4dee-8f1c-7d6a5b15f110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:00.605865 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90833c7-efd4-4dee-8f1c-7d6a5b15f110 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:00.606305 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90833c7-efd4-4dee-8f1c-7d6a5b15f110 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:00.606757 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90833c7-efd4-4dee-8f1c-7d6a5b15f110 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:00.607843 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90833c7-efd4-4dee-8f1c-7d6a5b15f110 None admin] RBAC: Adding query filter params (name=member) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:00.608299 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90833c7-efd4-4dee-8f1c-7d6a5b15f110 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:00.609969 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90833c7-efd4-4dee-8f1c-7d6a5b15f110 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:00.631315 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 71/133] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 6 16:54:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:00.637585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90833c7-efd4-4dee-8f1c-7d6a5b15f110 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:00.697926 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc62b7c-67e9-4d81-b801-36ecc1691205 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:00.718038 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:00.753566 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:00.802361 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e83978-ac1a-4577-8d56-e9fb41a3e27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:00.803762 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e83978-ac1a-4577-8d56-e9fb41a3e27e None admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:00.803762 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e83978-ac1a-4577-8d56-e9fb41a3e27e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:00.804115 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e83978-ac1a-4577-8d56-e9fb41a3e27e None admin] PATH_INFO: `/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1/users/8ad29aae43d24936a5dbec7f54883329/roles/c31c5ee7ed8646e48fa81e85d10fcaf2` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:00.809121 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e83978-ac1a-4577-8d56-e9fb41a3e27e None admin] RBAC: Authorizing `identity:create_grant(project_id=49b1ed5ea8fc417f8247e689e804e6f1, user_id=8ad29aae43d24936a5dbec7f54883329, role_id=c31c5ee7ed8646e48fa81e85d10fcaf2)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:00.810700 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e83978-ac1a-4577-8d56-e9fb41a3e27e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:00.838792 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90833c7-efd4-4dee-8f1c-7d6a5b15f110 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:00.879950 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129f78ec-d737-4e6f-8283-b15da4a4b60e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:00.881234 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f78ec-d737-4e6f-8283-b15da4a4b60e None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:00.881671 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f78ec-d737-4e6f-8283-b15da4a4b60e None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:00.882101 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129f78ec-d737-4e6f-8283-b15da4a4b60e None admin] PATH_INFO: `/v3/users/8ad29aae43d24936a5dbec7f54883329` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:00.883460 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129f78ec-d737-4e6f-8283-b15da4a4b60e None admin] RBAC: Authorizing `identity:get_user(user_id=8ad29aae43d24936a5dbec7f54883329)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:00.885161 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129f78ec-d737-4e6f-8283-b15da4a4b60e None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:00.887123 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 72/134] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:54:00 2022] GET /identity/v3/users/8ad29aae43d24936a5dbec7f54883329 => generated 275 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:00.888557 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad734c-c185-4cb3-b537-9f4a425843ce None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:00.889058 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad734c-c185-4cb3-b537-9f4a425843ce None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:00.889498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caad734c-c185-4cb3-b537-9f4a425843ce None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:00.890984 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 73/135] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:00.898381 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28048414-41e0-4ed9-95eb-6c9eeba94dd6 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:00.898917 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28048414-41e0-4ed9-95eb-6c9eeba94dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:00.899361 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28048414-41e0-4ed9-95eb-6c9eeba94dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:00.919507 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 63/136] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 6 16:54:00 2022] PUT /identity/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1/users/8ad29aae43d24936a5dbec7f54883329/roles/c31c5ee7ed8646e48fa81e85d10fcaf2 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 16:54:00.923576 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e83978-ac1a-4577-8d56-e9fb41a3e27e None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:00.923706 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-28048414-41e0-4ed9-95eb-6c9eeba94dd6 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:00.923706 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-28048414-41e0-4ed9-95eb-6c9eeba94dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:01.020789 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 74/137] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:01.064697 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28048414-41e0-4ed9-95eb-6c9eeba94dd6 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:01.069224 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e83978-ac1a-4577-8d56-e9fb41a3e27e None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:01.105691 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c29020-0719-47c5-9cd1-99a976d75010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:01.106572 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c29020-0719-47c5-9cd1-99a976d75010 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:01.106860 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c29020-0719-47c5-9cd1-99a976d75010 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:01.107139 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c29020-0719-47c5-9cd1-99a976d75010 None admin] PATH_INFO: `/v3/projects/a1142bf71f5b4e6e8c84cd2a9330b15d` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:01.112484 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c29020-0719-47c5-9cd1-99a976d75010 None admin] RBAC: Authorizing `identity:get_project(project_id=a1142bf71f5b4e6e8c84cd2a9330b15d)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:01.113987 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c29020-0719-47c5-9cd1-99a976d75010 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:01.116038 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 64/138] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:54:00 2022] GET /identity/v3/projects/a1142bf71f5b4e6e8c84cd2a9330b15d => generated 312 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:01.124765 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c29020-0719-47c5-9cd1-99a976d75010 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:01.132123 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:01.229352 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:01.273900 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28048414-41e0-4ed9-95eb-6c9eeba94dd6 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:01.289542 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c29020-0719-47c5-9cd1-99a976d75010 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:01.331618 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f565b-97c1-4f01-bf49-0a007fe0ec07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:01.332376 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f565b-97c1-4f01-bf49-0a007fe0ec07 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:01.332668 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f565b-97c1-4f01-bf49-0a007fe0ec07 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:01.332946 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f565b-97c1-4f01-bf49-0a007fe0ec07 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:01.334236 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458f565b-97c1-4f01-bf49-0a007fe0ec07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:01.335695 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458f565b-97c1-4f01-bf49-0a007fe0ec07 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:01.338727 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 75/139] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:54:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:01.346096 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f565b-97c1-4f01-bf49-0a007fe0ec07 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:01.368131 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8be6ff-bc5b-4e6a-8832-1607cfe86ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:01.369046 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8be6ff-bc5b-4e6a-8832-1607cfe86ac4 None admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:01.369337 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8be6ff-bc5b-4e6a-8832-1607cfe86ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:01.369621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8be6ff-bc5b-4e6a-8832-1607cfe86ac4 None admin] PATH_INFO: `/v3/projects/a1142bf71f5b4e6e8c84cd2a9330b15d/users/8ad29aae43d24936a5dbec7f54883329/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:01.380359 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8be6ff-bc5b-4e6a-8832-1607cfe86ac4 None admin] RBAC: Authorizing `identity:create_grant(project_id=a1142bf71f5b4e6e8c84cd2a9330b15d, user_id=8ad29aae43d24936a5dbec7f54883329, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:01.381829 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8be6ff-bc5b-4e6a-8832-1607cfe86ac4 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:01.458777 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 65/140] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 6 16:54:01 2022] PUT /identity/v3/projects/a1142bf71f5b4e6e8c84cd2a9330b15d/users/8ad29aae43d24936a5dbec7f54883329/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 16:54:01.497087 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f565b-97c1-4f01-bf49-0a007fe0ec07 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:01.499671 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:01.534976 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976441b7-e609-4a49-9f2d-6f980162dbb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:01.535673 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976441b7-e609-4a49-9f2d-6f980162dbb5 None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:01.536009 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976441b7-e609-4a49-9f2d-6f980162dbb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:01.536291 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976441b7-e609-4a49-9f2d-6f980162dbb5 None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:01.537025 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976441b7-e609-4a49-9f2d-6f980162dbb5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:01.538493 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976441b7-e609-4a49-9f2d-6f980162dbb5 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:01.676638 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 76/141] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 6 16:54:01 2022] POST /identity/v3/users => generated 283 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 16:54:01.939551 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49155c-106c-4e33-8c7f-8f723f2daaec None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:01.939899 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49155c-106c-4e33-8c7f-8f723f2daaec None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:01.940097 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf49155c-106c-4e33-8c7f-8f723f2daaec None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:01.941464 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 66/142] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:01.947321 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e238e-9922-48f1-9952-0d8dbd4ec4d5 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:01.947627 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e238e-9922-48f1-9952-0d8dbd4ec4d5 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:01.947942 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e238e-9922-48f1-9952-0d8dbd4ec4d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:01.965552 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-d79e238e-9922-48f1-9952-0d8dbd4ec4d5 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:01.966751 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d79e238e-9922-48f1-9952-0d8dbd4ec4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:02.039035 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 77/143] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:02.062535 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49155c-106c-4e33-8c7f-8f723f2daaec None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:02.181394 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf49155c-106c-4e33-8c7f-8f723f2daaec None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:02.193467 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:02.223032 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d67972-433e-4b70-a7d3-c7167a55db04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:02.223885 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d67972-433e-4b70-a7d3-c7167a55db04 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:02.224230 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d67972-433e-4b70-a7d3-c7167a55db04 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:02.224514 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d67972-433e-4b70-a7d3-c7167a55db04 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:02.238474 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d67972-433e-4b70-a7d3-c7167a55db04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:02.246676 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-02d67972-433e-4b70-a7d3-c7167a55db04 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 16:54:02.246676 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:02.246676 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:02.246676 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:02.246676 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:02.246676 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:02.246676 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:02.246676 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:02.246676 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:02.246676 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:02.246676 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:02.246676 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:02.246676 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:02.246676 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:02.246676 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:02.246676 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:02.247562 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:02.247562 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:02.247562 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:02.247562 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:02.247562 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:02.247562 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:02.247562 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:02.247562 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:02.247562 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:02.247562 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:02.247562 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:02.247562 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:02.247562 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:02.247562 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:02.247562 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:02.247562 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:02.248403 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:02.248403 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:02.248403 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:02.248403 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:02.248403 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:02.248403 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:02.248403 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:02.248403 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:02.248403 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:02.248403 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:02.248403 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 16:54:02.248403 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:02.248403 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 67/144] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:54:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:02.253980 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e238e-9922-48f1-9952-0d8dbd4ec4d5 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:02.355491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:02.365511 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e238e-9922-48f1-9952-0d8dbd4ec4d5 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:02.402514 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122bfc5d-de95-4de0-a4f3-5bd477aebea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:02.403206 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122bfc5d-de95-4de0-a4f3-5bd477aebea7 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:02.403501 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122bfc5d-de95-4de0-a4f3-5bd477aebea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:02.403813 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122bfc5d-de95-4de0-a4f3-5bd477aebea7 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:02.404632 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122bfc5d-de95-4de0-a4f3-5bd477aebea7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:02.404961 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122bfc5d-de95-4de0-a4f3-5bd477aebea7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:02.406378 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122bfc5d-de95-4de0-a4f3-5bd477aebea7 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:02.418101 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 78/145] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 6 16:54:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:02.423486 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d67972-433e-4b70-a7d3-c7167a55db04 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:02.537878 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d67972-433e-4b70-a7d3-c7167a55db04 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:02.578773 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84513c0d-eab9-43b6-a620-171557c9c1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:02.579678 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84513c0d-eab9-43b6-a620-171557c9c1d2 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:02.580012 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84513c0d-eab9-43b6-a620-171557c9c1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:02.580300 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84513c0d-eab9-43b6-a620-171557c9c1d2 None admin] PATH_INFO: `/v3/users/025c05bc4a8341ddb109caab361c5d41` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:02.581528 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84513c0d-eab9-43b6-a620-171557c9c1d2 None admin] RBAC: Authorizing `identity:get_user(user_id=025c05bc4a8341ddb109caab361c5d41)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:02.583003 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84513c0d-eab9-43b6-a620-171557c9c1d2 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:02.585033 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 68/146] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:54:02 2022] GET /identity/v3/users/025c05bc4a8341ddb109caab361c5d41 => generated 289 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:02.590526 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122bfc5d-de95-4de0-a4f3-5bd477aebea7 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:02.701355 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122bfc5d-de95-4de0-a4f3-5bd477aebea7 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:02.748050 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaae172-c82f-45f4-8629-78cb7e95205e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:02.748937 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaae172-c82f-45f4-8629-78cb7e95205e None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:02.749231 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaae172-c82f-45f4-8629-78cb7e95205e None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:02.749517 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaae172-c82f-45f4-8629-78cb7e95205e None admin] PATH_INFO: `/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:02.750742 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaae172-c82f-45f4-8629-78cb7e95205e None admin] RBAC: Authorizing `identity:get_project(project_id=49b1ed5ea8fc417f8247e689e804e6f1)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:02.752375 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcaae172-c82f-45f4-8629-78cb7e95205e None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:02.754293 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 79/147] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:54:02 2022] GET /identity/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:02.759782 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84513c0d-eab9-43b6-a620-171557c9c1d2 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:02.799287 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:02.855465 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1542995-bf0c-4b6a-82cb-ba2f022b3552 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:02.855795 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1542995-bf0c-4b6a-82cb-ba2f022b3552 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:02.856060 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1542995-bf0c-4b6a-82cb-ba2f022b3552 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:02.857585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 80/148] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:02.865598 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f6078-e7b7-4763-8a3d-c771cb4d1bb3 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:02.865894 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f6078-e7b7-4763-8a3d-c771cb4d1bb3 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:02.866181 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718f6078-e7b7-4763-8a3d-c771cb4d1bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:02.886375 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84513c0d-eab9-43b6-a620-171557c9c1d2 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:02.889723 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-718f6078-e7b7-4763-8a3d-c771cb4d1bb3 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:02.891837 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-718f6078-e7b7-4763-8a3d-c771cb4d1bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:02.913735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:02.978133 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a055f51d-8143-45cc-a278-7060f61e3676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:02.979511 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055f51d-8143-45cc-a278-7060f61e3676 None admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:02.980120 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055f51d-8143-45cc-a278-7060f61e3676 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:02.980594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055f51d-8143-45cc-a278-7060f61e3676 None admin] PATH_INFO: `/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1/users/025c05bc4a8341ddb109caab361c5d41/roles/e9f69ea47fce47fab94435de5a29de5f` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:02.983286 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a055f51d-8143-45cc-a278-7060f61e3676 None admin] RBAC: Authorizing `identity:create_grant(project_id=49b1ed5ea8fc417f8247e689e804e6f1, user_id=025c05bc4a8341ddb109caab361c5d41, role_id=e9f69ea47fce47fab94435de5a29de5f)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:02.985006 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a055f51d-8143-45cc-a278-7060f61e3676 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:02.990230 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 81/149] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:02.991651 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69c73ad-81f9-4868-9f1a-faaf748ff374 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:02.991973 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69c73ad-81f9-4868-9f1a-faaf748ff374 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:02.992258 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69c73ad-81f9-4868-9f1a-faaf748ff374 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:02.993679 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 82/150] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:02.999490 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa66bd2-8872-4c9e-9696-a9ad34aaaaa3 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:03.006526 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa66bd2-8872-4c9e-9696-a9ad34aaaaa3 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:03.006526 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa66bd2-8872-4c9e-9696-a9ad34aaaaa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:03.018867 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-9fa66bd2-8872-4c9e-9696-a9ad34aaaaa3 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:03.020411 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-9fa66bd2-8872-4c9e-9696-a9ad34aaaaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:03.059393 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 69/151] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 6 16:54:02 2022] PUT /identity/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1/users/025c05bc4a8341ddb109caab361c5d41/roles/e9f69ea47fce47fab94435de5a29de5f => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 16:54:03.060680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a055f51d-8143-45cc-a278-7060f61e3676 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:03.162622 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 83/152] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 16:54:03.233337 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:03.239704 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a055f51d-8143-45cc-a278-7060f61e3676 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:03.254248 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa66bd2-8872-4c9e-9696-a9ad34aaaaa3 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:03.305255 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a5bf6-6d5c-4a40-a5ad-884d06fba5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:03.308552 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a5bf6-6d5c-4a40-a5ad-884d06fba5f9 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:03.310353 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a5bf6-6d5c-4a40-a5ad-884d06fba5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:03.311756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a5bf6-6d5c-4a40-a5ad-884d06fba5f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:03.336308 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4a5bf6-6d5c-4a40-a5ad-884d06fba5f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:03.348444 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-0a4a5bf6-6d5c-4a40-a5ad-884d06fba5f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:54:03.348444 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:03.348444 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:03.348444 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:03.348444 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:03.348444 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:03.348444 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:03.348444 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:03.348444 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:03.348444 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:03.348444 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:03.348444 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:03.348444 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:03.348444 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:03.348444 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:03.348444 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:03.349424 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:03.349424 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:03.349424 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:03.349424 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:03.349424 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:03.349424 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:03.349424 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:03.349424 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:03.349424 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:03.349424 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:03.349424 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:03.349424 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:03.349424 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:03.349424 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:03.349424 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:03.349424 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:03.350213 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:03.350213 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:03.350213 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:03.350213 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:03.350213 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:03.350213 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:03.350213 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:03.350213 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:03.350213 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:03.350213 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:03.350213 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:54:03.350213 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:03.354350 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 70/153] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 6 16:54:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:03.361813 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a5bf6-6d5c-4a40-a5ad-884d06fba5f9 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:03.406053 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:03.430534 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa66bd2-8872-4c9e-9696-a9ad34aaaaa3 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:03.483459 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989e7664-6a5b-4feb-b29c-1da1813b0670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:03.484300 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e7664-6a5b-4feb-b29c-1da1813b0670 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:03.484582 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e7664-6a5b-4feb-b29c-1da1813b0670 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:03.484855 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989e7664-6a5b-4feb-b29c-1da1813b0670 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:03.497967 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a5bf6-6d5c-4a40-a5ad-884d06fba5f9 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:03.500801 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989e7664-6a5b-4feb-b29c-1da1813b0670 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:03.510474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-989e7664-6a5b-4feb-b29c-1da1813b0670 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 16:54:03.510474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:03.510474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:03.510474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:03.510474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:03.510474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:03.510474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:03.510474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:03.510474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:03.510474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:03.510474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:03.510474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:03.510474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:03.510474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:03.510474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:03.510474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:03.511627 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:03.511627 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:03.511627 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:03.511627 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:03.511627 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:03.511627 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:03.511627 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:03.511627 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:03.511627 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:03.511627 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:03.511627 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:03.511627 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:03.511627 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:03.511627 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:03.511627 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:03.511627 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:03.512573 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:03.512573 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:03.512573 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:03.512573 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:03.512573 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:03.512573 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:03.512573 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:03.512573 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:03.512573 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:03.512573 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:03.512573 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 16:54:03.512573 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:03.512573 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 84/154] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:54:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:03.518705 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989e7664-6a5b-4feb-b29c-1da1813b0670 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:03.542940 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e01ecd-a763-4ff5-b211-37c649b0d4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:03.544869 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e01ecd-a763-4ff5-b211-37c649b0d4d0 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:03.545395 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e01ecd-a763-4ff5-b211-37c649b0d4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:03.545888 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e01ecd-a763-4ff5-b211-37c649b0d4d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:03.546859 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e01ecd-a763-4ff5-b211-37c649b0d4d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:03.547921 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e01ecd-a763-4ff5-b211-37c649b0d4d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:03.549671 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e01ecd-a763-4ff5-b211-37c649b0d4d0 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:03.563563 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 71/155] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:54:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:03.568999 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e01ecd-a763-4ff5-b211-37c649b0d4d0 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:03.603278 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:03.644922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989e7664-6a5b-4feb-b29c-1da1813b0670 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:03.684905 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abf94a7-7980-46ea-977d-20f70e9c780a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:03.685589 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf94a7-7980-46ea-977d-20f70e9c780a None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:03.685871 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf94a7-7980-46ea-977d-20f70e9c780a None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:03.686145 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abf94a7-7980-46ea-977d-20f70e9c780a None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:03.686900 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abf94a7-7980-46ea-977d-20f70e9c780a None admin] RBAC: Adding query filter params (name=member) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:03.687261 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abf94a7-7980-46ea-977d-20f70e9c780a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:03.688940 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3abf94a7-7980-46ea-977d-20f70e9c780a None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:03.704746 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 85/156] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 6 16:54:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:03.715636 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abf94a7-7980-46ea-977d-20f70e9c780a None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:03.731643 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:03.767844 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e01ecd-a763-4ff5-b211-37c649b0d4d0 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:03.820634 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f41c1f1-7919-40c9-9431-92596a7de8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:03.821444 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f41c1f1-7919-40c9-9431-92596a7de8fe None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:03.821727 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f41c1f1-7919-40c9-9431-92596a7de8fe None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:03.822004 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f41c1f1-7919-40c9-9431-92596a7de8fe None admin] PATH_INFO: `/v3/users/fb49b9dd86cd415bb41412ca15c4d0ba` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:03.823438 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f41c1f1-7919-40c9-9431-92596a7de8fe None admin] RBAC: Authorizing `identity:get_user(user_id=fb49b9dd86cd415bb41412ca15c4d0ba)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:03.825131 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f41c1f1-7919-40c9-9431-92596a7de8fe None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:03.827782 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 72/157] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:54:03 2022] GET /identity/v3/users/fb49b9dd86cd415bb41412ca15c4d0ba => generated 247 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:03.832392 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f41c1f1-7919-40c9-9431-92596a7de8fe None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:03.938854 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:03.944253 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abf94a7-7980-46ea-977d-20f70e9c780a None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:03.959875 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f41c1f1-7919-40c9-9431-92596a7de8fe None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:04.008280 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a17d5-e985-4987-93d1-da57dafa1aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:04.008938 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a17d5-e985-4987-93d1-da57dafa1aaf None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:04.009268 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a17d5-e985-4987-93d1-da57dafa1aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:04.009543 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39a17d5-e985-4987-93d1-da57dafa1aaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:04.027789 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442296c8-c4f3-4b6e-9925-d0df62040b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:04.029126 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442296c8-c4f3-4b6e-9925-d0df62040b27 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:04.029612 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442296c8-c4f3-4b6e-9925-d0df62040b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:04.030083 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442296c8-c4f3-4b6e-9925-d0df62040b27 None admin] PATH_INFO: `/v3/users/8ad29aae43d24936a5dbec7f54883329` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:04.032106 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442296c8-c4f3-4b6e-9925-d0df62040b27 None admin] RBAC: Authorizing `identity:get_user(user_id=8ad29aae43d24936a5dbec7f54883329)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:04.033878 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442296c8-c4f3-4b6e-9925-d0df62040b27 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:04.036405 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 86/158] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:54:03 2022] GET /identity/v3/users/8ad29aae43d24936a5dbec7f54883329 => generated 275 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:04.061984 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442296c8-c4f3-4b6e-9925-d0df62040b27 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:04.111833 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39a17d5-e985-4987-93d1-da57dafa1aaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:04.113388 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39a17d5-e985-4987-93d1-da57dafa1aaf None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:04.234682 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:04.248050 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:04.251089 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 73/159] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:54:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 06 16:54:04.256718 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a17d5-e985-4987-93d1-da57dafa1aaf None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:04.320824 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442296c8-c4f3-4b6e-9925-d0df62040b27 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:04.393691 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39a17d5-e985-4987-93d1-da57dafa1aaf None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:04.414203 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ab2bc-ccdc-4336-8755-b71d50255185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:04.414955 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ab2bc-ccdc-4336-8755-b71d50255185 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:04.415332 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ab2bc-ccdc-4336-8755-b71d50255185 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:04.415683 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16ab2bc-ccdc-4336-8755-b71d50255185 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:04.445884 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b178d-99dc-4abe-be0a-02fa3daa9285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:04.446922 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b178d-99dc-4abe-be0a-02fa3daa9285 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:04.447208 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b178d-99dc-4abe-be0a-02fa3daa9285 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:04.447484 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b178d-99dc-4abe-be0a-02fa3daa9285 None admin] PATH_INFO: `/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:04.450552 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b178d-99dc-4abe-be0a-02fa3daa9285 None admin] RBAC: Authorizing `identity:get_project(project_id=49b1ed5ea8fc417f8247e689e804e6f1)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:04.452035 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b178d-99dc-4abe-be0a-02fa3daa9285 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:04.453758 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 74/160] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:54:04 2022] GET /identity/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:04.459489 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b178d-99dc-4abe-be0a-02fa3daa9285 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:04.536053 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:04.580270 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16ab2bc-ccdc-4336-8755-b71d50255185 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:04.581916 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16ab2bc-ccdc-4336-8755-b71d50255185 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:04.585404 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b178d-99dc-4abe-be0a-02fa3daa9285 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:04.623978 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e4d15-b57b-4302-96e0-b198603e1f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:04.624801 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e4d15-b57b-4302-96e0-b198603e1f6d None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:04.625220 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e4d15-b57b-4302-96e0-b198603e1f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:04.625501 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e4d15-b57b-4302-96e0-b198603e1f6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:04.626346 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701e4d15-b57b-4302-96e0-b198603e1f6d None admin] RBAC: Adding query filter params (role.id=b41f8a7a58e64f749f2099ca45cc04a7, scope.project.id=49b1ed5ea8fc417f8247e689e804e6f1, user.id=fb49b9dd86cd415bb41412ca15c4d0ba) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:04.626817 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701e4d15-b57b-4302-96e0-b198603e1f6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:04.628761 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701e4d15-b57b-4302-96e0-b198603e1f6d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:04.635554 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 75/161] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:54:04 2022] GET /identity/v3/role_assignments?user.id=fb49b9dd86cd415bb41412ca15c4d0ba&scope.project.id=49b1ed5ea8fc417f8247e689e804e6f1&role.id=b41f8a7a58e64f749f2099ca45cc04a7 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:04.637584 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8fe9d8-76f5-4ac5-9520-4f77fb8b7791 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:04.637874 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8fe9d8-76f5-4ac5-9520-4f77fb8b7791 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:04.638145 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8fe9d8-76f5-4ac5-9520-4f77fb8b7791 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:04.639394 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 76/162] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:04.645622 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc5052-2dd8-4732-8954-9f961e428648 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:04.645929 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc5052-2dd8-4732-8954-9f961e428648 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:04.646371 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc5052-2dd8-4732-8954-9f961e428648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:04.667892 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-72fc5052-2dd8-4732-8954-9f961e428648 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:04.669097 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-72fc5052-2dd8-4732-8954-9f961e428648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:04.748282 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:04.826050 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 87/163] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:54:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 764 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 16:54:04.831328 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 77/164] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 16:54:04.832364 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ab2bc-ccdc-4336-8755-b71d50255185 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:04.874497 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc5052-2dd8-4732-8954-9f961e428648 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:04.970724 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:05.012017 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16ab2bc-ccdc-4336-8755-b71d50255185 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:05.061995 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfa1837-6fcc-4642-8eb3-6a74cc714f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:05.063130 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa1837-6fcc-4642-8eb3-6a74cc714f30 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:05.063522 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa1837-6fcc-4642-8eb3-6a74cc714f30 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:05.063931 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfa1837-6fcc-4642-8eb3-6a74cc714f30 None admin] PATH_INFO: `/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:05.065351 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfa1837-6fcc-4642-8eb3-6a74cc714f30 None admin] RBAC: Authorizing `identity:get_project(project_id=49b1ed5ea8fc417f8247e689e804e6f1)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:05.066900 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfa1837-6fcc-4642-8eb3-6a74cc714f30 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:05.068859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 88/165] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:54:04 2022] GET /identity/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1 => generated 298 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:05.072471 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc5052-2dd8-4732-8954-9f961e428648 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:05.075937 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfa1837-6fcc-4642-8eb3-6a74cc714f30 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:05.144458 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63362407-4b2b-40c9-8275-96b664596a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:05.145296 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63362407-4b2b-40c9-8275-96b664596a16 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:05.145582 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63362407-4b2b-40c9-8275-96b664596a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:05.145860 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63362407-4b2b-40c9-8275-96b664596a16 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:05.146968 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:05.180625 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63362407-4b2b-40c9-8275-96b664596a16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:05.189373 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-63362407-4b2b-40c9-8275-96b664596a16 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 16:54:05.189373 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:05.189373 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:05.189373 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:05.189373 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:05.189373 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:05.189373 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:05.189373 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:05.189373 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:05.189373 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:05.189373 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:05.189373 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:05.189373 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:05.189373 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:05.189373 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:05.189373 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:05.190234 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:05.190234 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:05.190234 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:05.190234 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:05.190234 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:05.190234 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:05.190234 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:05.190234 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:05.190234 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:05.190234 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:05.190234 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:05.190234 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:05.190234 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:05.190234 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:05.190234 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:05.190234 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:05.191186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:05.191186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:05.191186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:05.191186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:05.191186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:05.191186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:05.191186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:05.191186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:05.191186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:05.191186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:05.191186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 16:54:05.191186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:05.191186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 78/166] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 6 16:54:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:54:05.198234 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63362407-4b2b-40c9-8275-96b664596a16 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:05.262535 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfa1837-6fcc-4642-8eb3-6a74cc714f30 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:05.332260 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7175b-6495-48c5-b7f9-2af7f01e9e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:05.333070 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7175b-6495-48c5-b7f9-2af7f01e9e18 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:05.333367 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7175b-6495-48c5-b7f9-2af7f01e9e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:05.333659 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac7175b-6495-48c5-b7f9-2af7f01e9e18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:05.334518 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7175b-6495-48c5-b7f9-2af7f01e9e18 None admin] RBAC: Adding query filter params (role.id=7fd63d2d99254c00bb3501173643ee30, scope.project.id=49b1ed5ea8fc417f8247e689e804e6f1, user.id=8ad29aae43d24936a5dbec7f54883329) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:05.334841 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7175b-6495-48c5-b7f9-2af7f01e9e18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:05.336356 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac7175b-6495-48c5-b7f9-2af7f01e9e18 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:05.350409 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 89/167] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:54:05 2022] GET /identity/v3/role_assignments?user.id=8ad29aae43d24936a5dbec7f54883329&scope.project.id=49b1ed5ea8fc417f8247e689e804e6f1&role.id=7fd63d2d99254c00bb3501173643ee30 => generated 608 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:05.415089 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63362407-4b2b-40c9-8275-96b664596a16 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:05.428083 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:05.454150 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d28da-1394-4392-80ac-af15715fe660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:05.454904 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d28da-1394-4392-80ac-af15715fe660 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:05.455199 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d28da-1394-4392-80ac-af15715fe660 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:05.455478 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7d28da-1394-4392-80ac-af15715fe660 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:05.456297 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7d28da-1394-4392-80ac-af15715fe660 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:05.456607 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7d28da-1394-4392-80ac-af15715fe660 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:05.458128 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7d28da-1394-4392-80ac-af15715fe660 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:05.467637 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 79/168] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:54:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:05.473431 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7175b-6495-48c5-b7f9-2af7f01e9e18 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:05.596312 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac7175b-6495-48c5-b7f9-2af7f01e9e18 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:05.597900 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:05.644584 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23310fd-c9ef-460c-a825-ba6f99982b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:05.645526 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23310fd-c9ef-460c-a825-ba6f99982b65 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:05.647240 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23310fd-c9ef-460c-a825-ba6f99982b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:05.647531 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23310fd-c9ef-460c-a825-ba6f99982b65 None admin] PATH_INFO: `/v3/users/8ad29aae43d24936a5dbec7f54883329` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:05.648893 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23310fd-c9ef-460c-a825-ba6f99982b65 None admin] RBAC: Authorizing `identity:get_user(user_id=8ad29aae43d24936a5dbec7f54883329)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:05.650396 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23310fd-c9ef-460c-a825-ba6f99982b65 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:05.652316 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 90/169] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:54:05 2022] GET /identity/v3/users/8ad29aae43d24936a5dbec7f54883329 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:05.658068 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d28da-1394-4392-80ac-af15715fe660 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:05.775079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7d28da-1394-4392-80ac-af15715fe660 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:05.809863 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edcfe5e-ec35-4cd1-b7d9-6c9bb61606ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:05.810518 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edcfe5e-ec35-4cd1-b7d9-6c9bb61606ac None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:05.810851 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edcfe5e-ec35-4cd1-b7d9-6c9bb61606ac None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:05.811139 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edcfe5e-ec35-4cd1-b7d9-6c9bb61606ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:05.851023 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:05.880404 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edcfe5e-ec35-4cd1-b7d9-6c9bb61606ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:05.881895 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edcfe5e-ec35-4cd1-b7d9-6c9bb61606ac None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:05.993916 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 80/170] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:54:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 16:54:05.999992 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23310fd-c9ef-460c-a825-ba6f99982b65 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:06.110741 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23310fd-c9ef-460c-a825-ba6f99982b65 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:06.149232 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a96178-2159-4dce-a7bd-6168eb0262b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:06.150101 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a96178-2159-4dce-a7bd-6168eb0262b7 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:06.150383 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a96178-2159-4dce-a7bd-6168eb0262b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:06.150653 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a96178-2159-4dce-a7bd-6168eb0262b7 None admin] PATH_INFO: `/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:06.152436 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a96178-2159-4dce-a7bd-6168eb0262b7 None admin] RBAC: Authorizing `identity:get_project(project_id=49b1ed5ea8fc417f8247e689e804e6f1)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:06.153994 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a96178-2159-4dce-a7bd-6168eb0262b7 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:06.155885 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 91/171] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:54:05 2022] GET /identity/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:06.161536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edcfe5e-ec35-4cd1-b7d9-6c9bb61606ac None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:06.233512 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:06.249643 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737f739-159c-443c-88d9-8390936fb353 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:06.249934 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737f739-159c-443c-88d9-8390936fb353 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:06.250206 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737f739-159c-443c-88d9-8390936fb353 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:06.251697 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 92/172] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:06.252813 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2af04a-8b55-4e9d-b666-d517bdd5e88b None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:06.253060 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2af04a-8b55-4e9d-b666-d517bdd5e88b None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:06.253333 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2af04a-8b55-4e9d-b666-d517bdd5e88b None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:06.254554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 93/173] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:06.258222 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7132126f-8ec3-4573-8c03-7282442f0cfb None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:06.258501 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7132126f-8ec3-4573-8c03-7282442f0cfb None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:06.258765 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7132126f-8ec3-4573-8c03-7282442f0cfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:06.283784 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edcfe5e-ec35-4cd1-b7d9-6c9bb61606ac None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:06.290499 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-7132126f-8ec3-4573-8c03-7282442f0cfb None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:06.292062 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-7132126f-8ec3-4573-8c03-7282442f0cfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:06.316953 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ece6bd-2e53-4c72-a28b-abbb5823d329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:06.317617 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ece6bd-2e53-4c72-a28b-abbb5823d329 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:06.317893 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ece6bd-2e53-4c72-a28b-abbb5823d329 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:06.318175 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ece6bd-2e53-4c72-a28b-abbb5823d329 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:06.319011 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ece6bd-2e53-4c72-a28b-abbb5823d329 None admin] RBAC: Adding query filter params (role.id=c31c5ee7ed8646e48fa81e85d10fcaf2, scope.project.id=49b1ed5ea8fc417f8247e689e804e6f1, user.id=8ad29aae43d24936a5dbec7f54883329) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:06.319320 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ece6bd-2e53-4c72-a28b-abbb5823d329 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:06.320823 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ece6bd-2e53-4c72-a28b-abbb5823d329 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:06.328055 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 81/174] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:54:06 2022] GET /identity/v3/role_assignments?user.id=8ad29aae43d24936a5dbec7f54883329&scope.project.id=49b1ed5ea8fc417f8247e689e804e6f1&role.id=c31c5ee7ed8646e48fa81e85d10fcaf2 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:06.329264 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e1687b-04eb-4f4c-af89-dd7276a6c528 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:06.329550 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e1687b-04eb-4f4c-af89-dd7276a6c528 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:06.329830 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e1687b-04eb-4f4c-af89-dd7276a6c528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:06.369053 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-c5e1687b-04eb-4f4c-af89-dd7276a6c528 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:06.370241 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c5e1687b-04eb-4f4c-af89-dd7276a6c528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:06.371799 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:06.461218 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 94/175] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:06.511360 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 82/176] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:06.516734 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7132126f-8ec3-4573-8c03-7282442f0cfb None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:06.552513 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e1687b-04eb-4f4c-af89-dd7276a6c528 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:06.643416 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:06.689185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e1687b-04eb-4f4c-af89-dd7276a6c528 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:06.743699 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e0353-69c2-42d1-8005-9e5e1c58dd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:06.744733 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e0353-69c2-42d1-8005-9e5e1c58dd77 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:06.745725 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e0353-69c2-42d1-8005-9e5e1c58dd77 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:06.746153 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e0353-69c2-42d1-8005-9e5e1c58dd77 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:06.746264 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7132126f-8ec3-4573-8c03-7282442f0cfb None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:06.761676 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e0353-69c2-42d1-8005-9e5e1c58dd77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:06.773405 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-fd9e0353-69c2-42d1-8005-9e5e1c58dd77 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:54:06.773405 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:06.773405 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:06.773405 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:06.773405 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:06.773405 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:06.773405 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:06.773405 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:06.773405 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:06.773405 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:06.773405 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:06.773405 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:06.773405 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:06.773405 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:06.773405 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:06.773405 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:06.774569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:06.774569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:06.774569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:06.774569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:06.774569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:06.774569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:06.774569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:06.774569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:06.774569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:06.774569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:06.774569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:06.774569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:06.774569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:06.774569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:06.774569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:06.774569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:06.775550 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:06.775550 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:06.775550 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:06.775550 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:06.775550 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:06.775550 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:06.775550 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:06.775550 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:06.775550 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:06.775550 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:06.775550 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:54:06.775550 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:06.778517 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 83/177] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 6 16:54:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:06.784325 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e0353-69c2-42d1-8005-9e5e1c58dd77 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:06.844436 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb09cf-5fe6-4670-bc38-0ddcbb395601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:06.845666 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb09cf-5fe6-4670-bc38-0ddcbb395601 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:06.848620 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb09cf-5fe6-4670-bc38-0ddcbb395601 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:06.849066 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb09cf-5fe6-4670-bc38-0ddcbb395601 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:06.871697 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb09cf-5fe6-4670-bc38-0ddcbb395601 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:06.885359 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-1dbb09cf-5fe6-4670-bc38-0ddcbb395601 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 16:54:06.885359 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:06.885359 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:06.885359 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:06.885359 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:06.885359 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:06.885359 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:06.885359 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:06.885359 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:06.885359 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:06.885359 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:06.885359 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:06.885359 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:06.885359 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:06.885359 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:06.885359 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:06.886981 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:06.886981 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:06.886981 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:06.886981 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:06.886981 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:06.886981 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:06.886981 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:06.886981 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:06.886981 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:06.886981 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:06.886981 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:06.886981 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:06.886981 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:06.886981 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:06.886981 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:06.886981 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:06.887800 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:06.887800 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:06.887800 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:06.887800 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:06.887800 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:06.887800 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:06.887800 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:06.887800 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:06.887800 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:06.887800 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:06.887800 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 16:54:06.887800 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:06.892343 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 95/178] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:54:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:06.898422 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb09cf-5fe6-4670-bc38-0ddcbb395601 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:06.946244 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:06.952417 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e0353-69c2-42d1-8005-9e5e1c58dd77 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:06.977277 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:06.988264 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dd1c6a-f0ff-4eb2-89b5-18cf8d64d829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:06.988940 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd1c6a-f0ff-4eb2-89b5-18cf8d64d829 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:06.989233 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd1c6a-f0ff-4eb2-89b5-18cf8d64d829 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:06.989489 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dd1c6a-f0ff-4eb2-89b5-18cf8d64d829 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:06.990239 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dd1c6a-f0ff-4eb2-89b5-18cf8d64d829 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:06.990545 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dd1c6a-f0ff-4eb2-89b5-18cf8d64d829 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:06.991969 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dd1c6a-f0ff-4eb2-89b5-18cf8d64d829 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:07.001838 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 84/179] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:54:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:07.003069 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb64e3-513a-4db9-bcb7-1b5241140b14 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:07.003360 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb64e3-513a-4db9-bcb7-1b5241140b14 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:07.003631 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fb64e3-513a-4db9-bcb7-1b5241140b14 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:07.008803 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 85/180] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:07.008803 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09490e7b-f941-4165-b272-400e77cf4c00 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:07.008803 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09490e7b-f941-4165-b272-400e77cf4c00 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:07.008803 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09490e7b-f941-4165-b272-400e77cf4c00 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:07.008803 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 86/181] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:07.012615 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09490e7b-f941-4165-b272-400e77cf4c00 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:07.031397 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb09cf-5fe6-4670-bc38-0ddcbb395601 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:07.069021 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd3473c-73e1-404a-9c42-144531cc023a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:07.069690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3473c-73e1-404a-9c42-144531cc023a None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:07.069970 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3473c-73e1-404a-9c42-144531cc023a None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:07.070303 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd3473c-73e1-404a-9c42-144531cc023a None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:07.071049 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd3473c-73e1-404a-9c42-144531cc023a None admin] RBAC: Adding query filter params (name=member) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:07.071493 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd3473c-73e1-404a-9c42-144531cc023a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:07.073090 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd3473c-73e1-404a-9c42-144531cc023a None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:07.086687 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 96/182] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 6 16:54:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:07.086687 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a25a6-bc99-4a70-a598-df3b9d70432c None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:07.086687 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a25a6-bc99-4a70-a598-df3b9d70432c None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:07.086687 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177a25a6-bc99-4a70-a598-df3b9d70432c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:07.101077 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-177a25a6-bc99-4a70-a598-df3b9d70432c None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:07.102285 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-177a25a6-bc99-4a70-a598-df3b9d70432c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:07.121051 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09490e7b-f941-4165-b272-400e77cf4c00 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:07.154305 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf57528-6931-4604-bd3a-1543a4088be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:07.155179 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf57528-6931-4604-bd3a-1543a4088be9 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:07.155464 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf57528-6931-4604-bd3a-1543a4088be9 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:07.155754 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf57528-6931-4604-bd3a-1543a4088be9 None admin] PATH_INFO: `/v3/users/0e523b449ff64dc59804790956bb6524` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:07.169281 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf57528-6931-4604-bd3a-1543a4088be9 None admin] RBAC: Authorizing `identity:get_user(user_id=0e523b449ff64dc59804790956bb6524)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:07.169567 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 97/183] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:07.170750 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf57528-6931-4604-bd3a-1543a4088be9 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:07.172520 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 87/184] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:54:07 2022] GET /identity/v3/users/0e523b449ff64dc59804790956bb6524 => generated 283 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:07.173256 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c2ec5-1f9a-48a1-be32-2928568eb2b9 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:07.173486 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf57528-6931-4604-bd3a-1543a4088be9 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:07.173935 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c2ec5-1f9a-48a1-be32-2928568eb2b9 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:07.174364 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c2ec5-1f9a-48a1-be32-2928568eb2b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:07.193295 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-ce3c2ec5-1f9a-48a1-be32-2928568eb2b9 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:07.194469 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-ce3c2ec5-1f9a-48a1-be32-2928568eb2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:07.208181 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:07.224687 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:07.259893 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 98/185] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:07.261412 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffbb4c7-dacd-4d75-8b56-47bd0a41e8ff None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:07.261702 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffbb4c7-dacd-4d75-8b56-47bd0a41e8ff None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:07.261979 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffbb4c7-dacd-4d75-8b56-47bd0a41e8ff None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:07.263254 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 99/186] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:07.263967 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffbb4c7-dacd-4d75-8b56-47bd0a41e8ff None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:07.277939 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf57528-6931-4604-bd3a-1543a4088be9 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:07.317865 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4d351-7131-488b-bf08-0f8f02ba3cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:07.318748 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4d351-7131-488b-bf08-0f8f02ba3cdb None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:07.319035 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4d351-7131-488b-bf08-0f8f02ba3cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:07.319316 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c4d351-7131-488b-bf08-0f8f02ba3cdb None admin] PATH_INFO: `/v3/users/8ad29aae43d24936a5dbec7f54883329` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:07.320827 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c4d351-7131-488b-bf08-0f8f02ba3cdb None admin] RBAC: Authorizing `identity:get_user(user_id=8ad29aae43d24936a5dbec7f54883329)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:07.322279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c4d351-7131-488b-bf08-0f8f02ba3cdb None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:07.324112 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 88/187] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:54:07 2022] GET /identity/v3/users/8ad29aae43d24936a5dbec7f54883329 => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:07.324712 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4d351-7131-488b-bf08-0f8f02ba3cdb None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:07.377842 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffbb4c7-dacd-4d75-8b56-47bd0a41e8ff None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:07.414025 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd66fc18-bed8-4159-b273-2ff3723238fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:07.414606 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66fc18-bed8-4159-b273-2ff3723238fe None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:07.414895 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66fc18-bed8-4159-b273-2ff3723238fe None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:07.415178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66fc18-bed8-4159-b273-2ff3723238fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:07.422343 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c4d351-7131-488b-bf08-0f8f02ba3cdb None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:07.433865 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:07.435412 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:07.455249 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863de12c-64fa-4842-a6c1-695ae4832c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:07.456083 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863de12c-64fa-4842-a6c1-695ae4832c8b None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:07.456373 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863de12c-64fa-4842-a6c1-695ae4832c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:07.456653 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863de12c-64fa-4842-a6c1-695ae4832c8b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:07.467650 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863de12c-64fa-4842-a6c1-695ae4832c8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:07.475274 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-863de12c-64fa-4842-a6c1-695ae4832c8b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 16:54:07.475274 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:07.475274 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:07.475274 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:07.475274 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:07.475274 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:07.475274 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:07.475274 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:07.475274 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:07.475274 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:07.475274 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:07.475274 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:07.475274 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:07.475274 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:07.475274 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:07.475274 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:07.480573 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:07.480573 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:07.480573 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:07.480573 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:07.480573 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:07.480573 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:07.480573 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:07.480573 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:07.480573 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:07.480573 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:07.480573 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:07.480573 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:07.480573 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:07.480573 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:07.480573 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:07.480573 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:07.481390 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:07.481390 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:07.481390 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:07.481390 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:07.481390 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:07.481390 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:07.481390 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:07.481390 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:07.481390 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:07.481390 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:07.481390 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 16:54:07.481390 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:07.481390 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 89/188] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:54:07 2022] GET /identity/v3/roles/reader => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:07.481390 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d5087-1a0d-4ea9-b701-8cb94a72b44a None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:07.482086 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d5087-1a0d-4ea9-b701-8cb94a72b44a None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:07.482086 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d5087-1a0d-4ea9-b701-8cb94a72b44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:07.486651 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd66fc18-bed8-4159-b273-2ff3723238fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:07.488169 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd66fc18-bed8-4159-b273-2ff3723238fe None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:07.494404 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-685d5087-1a0d-4ea9-b701-8cb94a72b44a None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:07.495590 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-685d5087-1a0d-4ea9-b701-8cb94a72b44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:07.561487 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 90/189] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:07.562337 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685d5087-1a0d-4ea9-b701-8cb94a72b44a None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:07.586819 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 100/190] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:54:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 16:54:07.588323 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd66fc18-bed8-4159-b273-2ff3723238fe None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:07.638904 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:07.666397 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685d5087-1a0d-4ea9-b701-8cb94a72b44a None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:07.667854 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:07.687305 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd66fc18-bed8-4159-b273-2ff3723238fe None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:07.700695 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb5ba1d-f1d7-45c9-bd5a-acea7080a9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:07.702167 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5ba1d-f1d7-45c9-bd5a-acea7080a9f2 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:07.702167 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5ba1d-f1d7-45c9-bd5a-acea7080a9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:07.702167 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb5ba1d-f1d7-45c9-bd5a-acea7080a9f2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:07.703220 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb5ba1d-f1d7-45c9-bd5a-acea7080a9f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:07.704710 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb5ba1d-f1d7-45c9-bd5a-acea7080a9f2 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:07.706317 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 91/191] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:54:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:07.707090 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb5ba1d-f1d7-45c9-bd5a-acea7080a9f2 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:07.721867 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1877f6a-8403-44ae-b3bf-22a9e3f1a02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:07.722522 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1877f6a-8403-44ae-b3bf-22a9e3f1a02c None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:07.722816 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1877f6a-8403-44ae-b3bf-22a9e3f1a02c None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:07.723094 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1877f6a-8403-44ae-b3bf-22a9e3f1a02c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:07.788614 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1877f6a-8403-44ae-b3bf-22a9e3f1a02c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:07.790061 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1877f6a-8403-44ae-b3bf-22a9e3f1a02c None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:07.804942 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb5ba1d-f1d7-45c9-bd5a-acea7080a9f2 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:07.837608 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:07.837775 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf61ab-2b0e-4149-a5d4-f677f3726d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:07.838359 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf61ab-2b0e-4149-a5d4-f677f3726d53 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:07.838640 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf61ab-2b0e-4149-a5d4-f677f3726d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:07.838965 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bf61ab-2b0e-4149-a5d4-f677f3726d53 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:07.839700 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bf61ab-2b0e-4149-a5d4-f677f3726d53 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:07.840032 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bf61ab-2b0e-4149-a5d4-f677f3726d53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:07.841481 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bf61ab-2b0e-4149-a5d4-f677f3726d53 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:07.849585 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 92/192] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 6 16:54:07 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:07.854320 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf61ab-2b0e-4149-a5d4-f677f3726d53 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:07.882500 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:07.892957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 101/193] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:54:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 16:54:07.892957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1877f6a-8403-44ae-b3bf-22a9e3f1a02c None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:07.954296 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bf61ab-2b0e-4149-a5d4-f677f3726d53 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:07.990822 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f581dcde-895e-4b9f-aa3b-a00d3100d56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:07.991636 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f581dcde-895e-4b9f-aa3b-a00d3100d56c None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:07.991950 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f581dcde-895e-4b9f-aa3b-a00d3100d56c None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:07.992224 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f581dcde-895e-4b9f-aa3b-a00d3100d56c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:07.994693 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1877f6a-8403-44ae-b3bf-22a9e3f1a02c None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:08.003542 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f581dcde-895e-4b9f-aa3b-a00d3100d56c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:08.011670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-f581dcde-895e-4b9f-aa3b-a00d3100d56c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 16:54:08.011670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:08.011670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:08.011670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:08.011670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:08.011670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:08.011670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:08.011670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:08.011670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:08.011670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:08.011670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:08.011670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:08.011670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:08.011670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:08.011670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:08.011670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:08.012747 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:08.012747 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:08.012747 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:08.012747 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:08.012747 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:08.012747 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:08.012747 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:08.012747 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:08.012747 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:08.012747 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:08.012747 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:08.012747 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:08.012747 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:08.012747 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:08.012747 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:08.012747 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:08.013722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:08.013722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:08.013722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:08.013722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:08.013722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:08.013722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:08.013722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:08.013722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:08.013722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:08.013722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:08.013722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 16:54:08.013722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:08.013722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 93/194] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 6 16:54:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:08.014514 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f581dcde-895e-4b9f-aa3b-a00d3100d56c None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:08.032368 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db3646-92f6-4b63-85b7-b859ff37ff45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:08.033347 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db3646-92f6-4b63-85b7-b859ff37ff45 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:08.033642 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db3646-92f6-4b63-85b7-b859ff37ff45 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:08.033923 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db3646-92f6-4b63-85b7-b859ff37ff45 None admin] PATH_INFO: `/v3/projects/aa8007020ec543c1a6405283ddd21067` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:08.035122 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8db3646-92f6-4b63-85b7-b859ff37ff45 None admin] RBAC: Authorizing `identity:get_project(project_id=aa8007020ec543c1a6405283ddd21067)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:08.036651 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8db3646-92f6-4b63-85b7-b859ff37ff45 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:08.038354 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 102/195] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:54:07 2022] GET /identity/v3/projects/aa8007020ec543c1a6405283ddd21067 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:08.039111 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db3646-92f6-4b63-85b7-b859ff37ff45 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:08.059296 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:08.108010 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:08.116669 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f581dcde-895e-4b9f-aa3b-a00d3100d56c None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:08.137332 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db3646-92f6-4b63-85b7-b859ff37ff45 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:08.150284 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27072888-0d6a-4362-adea-81a4473a4855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:08.150955 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27072888-0d6a-4362-adea-81a4473a4855 None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:08.151242 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27072888-0d6a-4362-adea-81a4473a4855 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:08.152061 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27072888-0d6a-4362-adea-81a4473a4855 None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:08.152778 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27072888-0d6a-4362-adea-81a4473a4855 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:08.154225 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27072888-0d6a-4362-adea-81a4473a4855 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:08.173148 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e754624b-4277-4ecb-b6bc-a4f34a02f129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:08.174019 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e754624b-4277-4ecb-b6bc-a4f34a02f129 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:08.174305 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e754624b-4277-4ecb-b6bc-a4f34a02f129 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:08.174582 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e754624b-4277-4ecb-b6bc-a4f34a02f129 None admin] PATH_INFO: `/v3/users/025c05bc4a8341ddb109caab361c5d41` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:08.175796 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e754624b-4277-4ecb-b6bc-a4f34a02f129 None admin] RBAC: Authorizing `identity:get_user(user_id=025c05bc4a8341ddb109caab361c5d41)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:08.177241 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e754624b-4277-4ecb-b6bc-a4f34a02f129 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:08.178874 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 103/196] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:54:08 2022] GET /identity/v3/users/025c05bc4a8341ddb109caab361c5d41 => generated 289 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:08.179633 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e754624b-4277-4ecb-b6bc-a4f34a02f129 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:08.262552 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:08.264699 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 94/197] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 6 16:54:08 2022] POST /identity/v3/users => generated 297 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 16:54:08.264699 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27072888-0d6a-4362-adea-81a4473a4855 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:08.280738 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e754624b-4277-4ecb-b6bc-a4f34a02f129 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:08.322634 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16921374-1842-4dad-9100-5be107a2cc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:08.323580 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16921374-1842-4dad-9100-5be107a2cc67 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:08.324018 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16921374-1842-4dad-9100-5be107a2cc67 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:08.324299 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16921374-1842-4dad-9100-5be107a2cc67 None admin] PATH_INFO: `/v3/projects/a1142bf71f5b4e6e8c84cd2a9330b15d` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:08.325548 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16921374-1842-4dad-9100-5be107a2cc67 None admin] RBAC: Authorizing `identity:get_project(project_id=a1142bf71f5b4e6e8c84cd2a9330b15d)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:08.327238 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16921374-1842-4dad-9100-5be107a2cc67 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:08.329306 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 104/198] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:54:08 2022] GET /identity/v3/projects/a1142bf71f5b4e6e8c84cd2a9330b15d => generated 312 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:08.329939 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16921374-1842-4dad-9100-5be107a2cc67 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:08.380857 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27072888-0d6a-4362-adea-81a4473a4855 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:08.421765 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fbd5f8-62bb-4280-95dd-9cd9c12b8a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:08.423148 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fbd5f8-62bb-4280-95dd-9cd9c12b8a2e None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:08.423148 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fbd5f8-62bb-4280-95dd-9cd9c12b8a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:08.423148 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fbd5f8-62bb-4280-95dd-9cd9c12b8a2e None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:08.424578 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fbd5f8-62bb-4280-95dd-9cd9c12b8a2e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:08.424578 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fbd5f8-62bb-4280-95dd-9cd9c12b8a2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:08.425741 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fbd5f8-62bb-4280-95dd-9cd9c12b8a2e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:08.434770 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 95/199] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:54:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:08.435715 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fbd5f8-62bb-4280-95dd-9cd9c12b8a2e None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:08.445515 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16921374-1842-4dad-9100-5be107a2cc67 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:08.449595 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:08.487912 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c99ca-3a36-4b59-b8a5-e5f5e0cb77dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:08.488576 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c99ca-3a36-4b59-b8a5-e5f5e0cb77dd None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:08.488859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c99ca-3a36-4b59-b8a5-e5f5e0cb77dd None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:08.489131 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c99ca-3a36-4b59-b8a5-e5f5e0cb77dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:08.489969 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2c99ca-3a36-4b59-b8a5-e5f5e0cb77dd None admin] RBAC: Adding query filter params (role.id=b41f8a7a58e64f749f2099ca45cc04a7, scope.project.id=aa8007020ec543c1a6405283ddd21067, user.id=0e523b449ff64dc59804790956bb6524) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:08.490281 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2c99ca-3a36-4b59-b8a5-e5f5e0cb77dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:08.491741 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2c99ca-3a36-4b59-b8a5-e5f5e0cb77dd None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:08.497527 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 105/200] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:54:08 2022] GET /identity/v3/role_assignments?user.id=0e523b449ff64dc59804790956bb6524&scope.project.id=aa8007020ec543c1a6405283ddd21067&role.id=b41f8a7a58e64f749f2099ca45cc04a7 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:08.498155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c99ca-3a36-4b59-b8a5-e5f5e0cb77dd None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:08.518533 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:08.558576 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fbd5f8-62bb-4280-95dd-9cd9c12b8a2e None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:08.607893 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384ffdb7-d141-4207-9fb7-9b3449fd4337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:08.608571 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ffdb7-d141-4207-9fb7-9b3449fd4337 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:08.608854 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ffdb7-d141-4207-9fb7-9b3449fd4337 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:08.609158 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384ffdb7-d141-4207-9fb7-9b3449fd4337 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:08.641107 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c99ca-3a36-4b59-b8a5-e5f5e0cb77dd None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:08.685975 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a96f2a-daf8-4322-b378-10292c894255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:08.686938 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a96f2a-daf8-4322-b378-10292c894255 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:08.687412 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a96f2a-daf8-4322-b378-10292c894255 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:08.687691 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a96f2a-daf8-4322-b378-10292c894255 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:08.688581 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a96f2a-daf8-4322-b378-10292c894255 None admin] RBAC: Adding query filter params (role.id=7fd63d2d99254c00bb3501173643ee30, scope.project.id=a1142bf71f5b4e6e8c84cd2a9330b15d, user.id=8ad29aae43d24936a5dbec7f54883329) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:08.688897 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a96f2a-daf8-4322-b378-10292c894255 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:08.690336 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a96f2a-daf8-4322-b378-10292c894255 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:08.698712 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 106/201] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:54:08 2022] GET /identity/v3/role_assignments?user.id=8ad29aae43d24936a5dbec7f54883329&scope.project.id=a1142bf71f5b4e6e8c84cd2a9330b15d&role.id=7fd63d2d99254c00bb3501173643ee30 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:08.698954 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a96f2a-daf8-4322-b378-10292c894255 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:08.725054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384ffdb7-d141-4207-9fb7-9b3449fd4337 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:08.728145 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384ffdb7-d141-4207-9fb7-9b3449fd4337 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:08.747647 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:08.811483 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:08.845074 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a96f2a-daf8-4322-b378-10292c894255 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:08.899392 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 96/202] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:54:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 16:54:08.905123 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384ffdb7-d141-4207-9fb7-9b3449fd4337 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:08.929180 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80940b4e-6986-48ed-9106-840c36865dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:08.930867 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80940b4e-6986-48ed-9106-840c36865dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:08.931371 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80940b4e-6986-48ed-9106-840c36865dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:08.931877 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80940b4e-6986-48ed-9106-840c36865dc1 None admin] PATH_INFO: `/v3/users/0e523b449ff64dc59804790956bb6524` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:08.934474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80940b4e-6986-48ed-9106-840c36865dc1 None admin] RBAC: Authorizing `identity:get_user(user_id=0e523b449ff64dc59804790956bb6524)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:08.936996 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80940b4e-6986-48ed-9106-840c36865dc1 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:08.939243 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 107/203] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:54:08 2022] GET /identity/v3/users/0e523b449ff64dc59804790956bb6524 => generated 283 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:08.949324 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80940b4e-6986-48ed-9106-840c36865dc1 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:09.049984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384ffdb7-d141-4207-9fb7-9b3449fd4337 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:09.069761 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:09.128342 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80940b4e-6986-48ed-9106-840c36865dc1 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:09.137773 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8976e414-6132-4c90-81fb-7f31b6e10fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:09.138788 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8976e414-6132-4c90-81fb-7f31b6e10fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:09.139165 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8976e414-6132-4c90-81fb-7f31b6e10fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:09.139789 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8976e414-6132-4c90-81fb-7f31b6e10fc0 None admin] PATH_INFO: `/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:09.144502 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8976e414-6132-4c90-81fb-7f31b6e10fc0 None admin] RBAC: Authorizing `identity:get_project(project_id=49b1ed5ea8fc417f8247e689e804e6f1)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:09.146396 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8976e414-6132-4c90-81fb-7f31b6e10fc0 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:09.151750 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 97/204] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:54:08 2022] GET /identity/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1 => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:09.172853 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8976e414-6132-4c90-81fb-7f31b6e10fc0 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:09.179655 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf15e3-4886-4933-aa2e-758da0353ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:09.180364 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf15e3-4886-4933-aa2e-758da0353ca1 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:09.180651 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf15e3-4886-4933-aa2e-758da0353ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:09.180935 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf15e3-4886-4933-aa2e-758da0353ca1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:09.205975 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:09.264585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbf15e3-4886-4933-aa2e-758da0353ca1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:09.266491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbf15e3-4886-4933-aa2e-758da0353ca1 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:09.381350 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 108/205] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:54:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 16:54:09.397333 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf15e3-4886-4933-aa2e-758da0353ca1 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:09.432550 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8976e414-6132-4c90-81fb-7f31b6e10fc0 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:09.466436 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:09.523895 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbf15e3-4886-4933-aa2e-758da0353ca1 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:09.548430 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ca402a-6ada-4345-b3b4-11fae8f6bb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:09.549114 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca402a-6ada-4345-b3b4-11fae8f6bb9e None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:09.549406 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca402a-6ada-4345-b3b4-11fae8f6bb9e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:09.549686 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ca402a-6ada-4345-b3b4-11fae8f6bb9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:09.550536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ca402a-6ada-4345-b3b4-11fae8f6bb9e None admin] RBAC: Adding query filter params (role.id=e9f69ea47fce47fab94435de5a29de5f, scope.project.id=49b1ed5ea8fc417f8247e689e804e6f1, user.id=025c05bc4a8341ddb109caab361c5d41) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:09.550848 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ca402a-6ada-4345-b3b4-11fae8f6bb9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:09.552332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ca402a-6ada-4345-b3b4-11fae8f6bb9e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:09.564639 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee07abf-df7c-4df7-a381-77f3b69a5c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:09.565496 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee07abf-df7c-4df7-a381-77f3b69a5c2f None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:09.565778 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee07abf-df7c-4df7-a381-77f3b69a5c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:09.566050 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee07abf-df7c-4df7-a381-77f3b69a5c2f None admin] PATH_INFO: `/v3/projects/aa8007020ec543c1a6405283ddd21067` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:09.568655 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 98/206] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:54:09 2022] GET /identity/v3/role_assignments?user.id=025c05bc4a8341ddb109caab361c5d41&scope.project.id=49b1ed5ea8fc417f8247e689e804e6f1&role.id=e9f69ea47fce47fab94435de5a29de5f => generated 608 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:09.572485 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee07abf-df7c-4df7-a381-77f3b69a5c2f None admin] RBAC: Authorizing `identity:get_project(project_id=aa8007020ec543c1a6405283ddd21067)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:09.574251 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee07abf-df7c-4df7-a381-77f3b69a5c2f None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:09.576359 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 109/207] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:54:09 2022] GET /identity/v3/projects/aa8007020ec543c1a6405283ddd21067 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:09.591394 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ca402a-6ada-4345-b3b4-11fae8f6bb9e None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:09.654217 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:09.732245 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ca402a-6ada-4345-b3b4-11fae8f6bb9e None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:09.779233 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be28cdd-9605-43f6-8cb7-4948e70fb2d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:09.780311 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be28cdd-9605-43f6-8cb7-4948e70fb2d1 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:09.780819 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be28cdd-9605-43f6-8cb7-4948e70fb2d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:09.781293 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be28cdd-9605-43f6-8cb7-4948e70fb2d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:09.782323 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be28cdd-9605-43f6-8cb7-4948e70fb2d1 None admin] RBAC: Adding query filter params (role.id=c31c5ee7ed8646e48fa81e85d10fcaf2, scope.project.id=aa8007020ec543c1a6405283ddd21067, user.id=0e523b449ff64dc59804790956bb6524) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:09.782830 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be28cdd-9605-43f6-8cb7-4948e70fb2d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:09.784515 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be28cdd-9605-43f6-8cb7-4948e70fb2d1 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:09.793719 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 99/208] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:54:09 2022] GET /identity/v3/role_assignments?user.id=0e523b449ff64dc59804790956bb6524&scope.project.id=aa8007020ec543c1a6405283ddd21067&role.id=c31c5ee7ed8646e48fa81e85d10fcaf2 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:10.756809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19618124-54c0-40e0-83e4-1a19f325f1aa None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:10.757274 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19618124-54c0-40e0-83e4-1a19f325f1aa None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:10.757380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19618124-54c0-40e0-83e4-1a19f325f1aa None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:10.759052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 110/209] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:10.764313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76894a7-265b-4bac-a36c-20e07db744f3 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:10.764699 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76894a7-265b-4bac-a36c-20e07db744f3 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:10.764884 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76894a7-265b-4bac-a36c-20e07db744f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:10.782493 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-d76894a7-265b-4bac-a36c-20e07db744f3 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:10.783688 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d76894a7-265b-4bac-a36c-20e07db744f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:10.826419 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08470bc-7da8-4063-af0e-ff5f1bfb79bc None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:10.826715 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08470bc-7da8-4063-af0e-ff5f1bfb79bc None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:10.826990 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08470bc-7da8-4063-af0e-ff5f1bfb79bc None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:10.828655 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 111/210] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:10.833418 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f20fd-6307-4bb3-8612-0e11a9390df0 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:10.833707 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f20fd-6307-4bb3-8612-0e11a9390df0 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:10.833987 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f20fd-6307-4bb3-8612-0e11a9390df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:10.850647 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-3d7f20fd-6307-4bb3-8612-0e11a9390df0 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:10.851874 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-3d7f20fd-6307-4bb3-8612-0e11a9390df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:10.856758 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 100/211] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:10.880479 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76894a7-265b-4bac-a36c-20e07db744f3 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:10.928194 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:10.930892 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 112/212] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 16:54:10.942661 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6f7da-856b-495b-852d-6dfe2b23b562 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:10.950552 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6f7da-856b-495b-852d-6dfe2b23b562 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:10.950552 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f6f7da-856b-495b-852d-6dfe2b23b562 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:10.950552 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 113/213] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 16:54:10.950823 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52afc8d4-709b-44eb-9030-082260dbaf74 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:10.950904 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52afc8d4-709b-44eb-9030-082260dbaf74 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:10.951274 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52afc8d4-709b-44eb-9030-082260dbaf74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:10.968269 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-52afc8d4-709b-44eb-9030-082260dbaf74 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:10.969475 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-52afc8d4-709b-44eb-9030-082260dbaf74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:11.002923 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76894a7-265b-4bac-a36c-20e07db744f3 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:11.015908 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:11.039968 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 114/214] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 16:54:11.040334 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42332397-0997-44de-97ce-2eaf037ac5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:11.041148 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42332397-0997-44de-97ce-2eaf037ac5c8 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:11.041430 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42332397-0997-44de-97ce-2eaf037ac5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:11.041708 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42332397-0997-44de-97ce-2eaf037ac5c8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:11.045158 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52afc8d4-709b-44eb-9030-082260dbaf74 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:11.053687 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42332397-0997-44de-97ce-2eaf037ac5c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:11.062381 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-42332397-0997-44de-97ce-2eaf037ac5c8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 16:54:11.062381 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:11.062381 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:11.062381 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:11.062381 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:11.062381 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:11.062381 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:11.062381 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:11.062381 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:11.062381 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:11.062381 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:11.062381 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:11.062381 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:11.062381 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:11.062381 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:11.062381 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:11.063313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:11.063313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:11.063313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:11.063313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:11.063313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:11.063313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:11.063313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:11.063313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:11.063313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:11.063313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:11.063313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:11.063313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:11.063313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:11.063313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:11.063313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:11.063313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:11.068656 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:11.068656 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:11.068656 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:11.068656 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:11.068656 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:11.068656 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:11.068656 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:11.068656 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:11.068656 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:11.068656 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:11.068656 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 16:54:11.068656 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:11.068656 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 101/215] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:54:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:54:11.068656 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42332397-0997-44de-97ce-2eaf037ac5c8 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:11.156042 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52afc8d4-709b-44eb-9030-082260dbaf74 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:11.166118 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:11.171771 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42332397-0997-44de-97ce-2eaf037ac5c8 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:11.193393 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddcdd54-c92b-4b3d-b66e-abb155a7e2ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:11.194303 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddcdd54-c92b-4b3d-b66e-abb155a7e2ea None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:11.194555 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddcdd54-c92b-4b3d-b66e-abb155a7e2ea None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:11.194882 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddcdd54-c92b-4b3d-b66e-abb155a7e2ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:11.196039 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddcdd54-c92b-4b3d-b66e-abb155a7e2ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:11.197496 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddcdd54-c92b-4b3d-b66e-abb155a7e2ea None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:11.199897 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 115/216] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:54:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:11.200474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddcdd54-c92b-4b3d-b66e-abb155a7e2ea None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:11.211357 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c75451f-cecf-4c4d-9352-cbd65871cf96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:11.212192 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75451f-cecf-4c4d-9352-cbd65871cf96 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:11.212475 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75451f-cecf-4c4d-9352-cbd65871cf96 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:11.212750 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75451f-cecf-4c4d-9352-cbd65871cf96 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:11.223491 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c75451f-cecf-4c4d-9352-cbd65871cf96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:11.231219 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-2c75451f-cecf-4c4d-9352-cbd65871cf96 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:54:11.231219 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:11.231219 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:11.231219 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:11.231219 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:11.231219 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:11.231219 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:11.231219 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:11.231219 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:11.231219 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:11.231219 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:11.231219 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:11.231219 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:11.231219 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:11.231219 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:11.231219 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:11.232184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:11.232184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:11.232184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:11.232184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:11.232184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:11.232184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:11.232184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:11.232184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:11.232184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:11.232184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:11.232184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:11.232184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:11.232184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:11.232184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:11.232184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:11.232184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:11.233083 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:11.233083 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:11.233083 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:11.233083 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:11.233083 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:11.233083 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:11.233083 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:11.233083 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:11.233083 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:11.233083 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:11.233083 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:54:11.233083 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:11.233083 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 102/217] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 6 16:54:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:11.233741 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c75451f-cecf-4c4d-9352-cbd65871cf96 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:11.255496 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:11.312221 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddcdd54-c92b-4b3d-b66e-abb155a7e2ea None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:11.347799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c75451f-cecf-4c4d-9352-cbd65871cf96 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:11.351386 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9716a8ee-da7a-44c9-913e-9528844d0c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:11.352094 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716a8ee-da7a-44c9-913e-9528844d0c37 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:11.352383 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716a8ee-da7a-44c9-913e-9528844d0c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:11.352667 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716a8ee-da7a-44c9-913e-9528844d0c37 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:11.353426 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9716a8ee-da7a-44c9-913e-9528844d0c37 None admin] RBAC: Adding query filter params (name=member) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:11.353735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9716a8ee-da7a-44c9-913e-9528844d0c37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:11.355647 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9716a8ee-da7a-44c9-913e-9528844d0c37 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:11.367771 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 116/218] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 6 16:54:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:11.367771 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9716a8ee-da7a-44c9-913e-9528844d0c37 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:11.389933 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd5183c-d257-49c2-8cb6-48aefdd498e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:11.390610 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5183c-d257-49c2-8cb6-48aefdd498e2 None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:11.390909 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5183c-d257-49c2-8cb6-48aefdd498e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:11.391178 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5183c-d257-49c2-8cb6-48aefdd498e2 None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:11.391919 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd5183c-d257-49c2-8cb6-48aefdd498e2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:11.393406 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd5183c-d257-49c2-8cb6-48aefdd498e2 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:11.469875 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9716a8ee-da7a-44c9-913e-9528844d0c37 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:11.479147 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:11.483251 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 103/219] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 6 16:54:11 2022] POST /identity/v3/users => generated 297 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 16:54:11.489813 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd5183c-d257-49c2-8cb6-48aefdd498e2 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:11.511715 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:11.517323 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98cfa07-c7f9-4581-925f-58d733737232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:11.518441 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98cfa07-c7f9-4581-925f-58d733737232 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:11.518732 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98cfa07-c7f9-4581-925f-58d733737232 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:11.519014 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98cfa07-c7f9-4581-925f-58d733737232 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:11.519890 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98cfa07-c7f9-4581-925f-58d733737232 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:11.520264 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98cfa07-c7f9-4581-925f-58d733737232 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:11.522094 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98cfa07-c7f9-4581-925f-58d733737232 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:11.540532 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 117/220] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:54:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:11.542847 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98cfa07-c7f9-4581-925f-58d733737232 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:11.624945 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd5183c-d257-49c2-8cb6-48aefdd498e2 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:11.655546 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98cfa07-c7f9-4581-925f-58d733737232 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:11.674236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f625e5e7-45ba-491f-bd79-5e4856974620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:11.675130 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625e5e7-45ba-491f-bd79-5e4856974620 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:11.675418 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625e5e7-45ba-491f-bd79-5e4856974620 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:11.675696 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625e5e7-45ba-491f-bd79-5e4856974620 None admin] PATH_INFO: `/v3/users/2ad8193e60d54439a0c591bb264dff53` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:11.693133 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f625e5e7-45ba-491f-bd79-5e4856974620 None admin] RBAC: Authorizing `identity:get_user(user_id=2ad8193e60d54439a0c591bb264dff53)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:11.694621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f625e5e7-45ba-491f-bd79-5e4856974620 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:11.700204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 104/221] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:54:11 2022] GET /identity/v3/users/2ad8193e60d54439a0c591bb264dff53 => generated 297 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:11.702880 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f625e5e7-45ba-491f-bd79-5e4856974620 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:11.708923 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaba70e-feb2-4c96-b3e0-c118d86b4f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:11.709865 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaba70e-feb2-4c96-b3e0-c118d86b4f5c None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:11.710153 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaba70e-feb2-4c96-b3e0-c118d86b4f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:11.710432 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaba70e-feb2-4c96-b3e0-c118d86b4f5c None admin] PATH_INFO: `/v3/users/0e523b449ff64dc59804790956bb6524` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:11.711790 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaba70e-feb2-4c96-b3e0-c118d86b4f5c None admin] RBAC: Authorizing `identity:get_user(user_id=0e523b449ff64dc59804790956bb6524)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:11.713259 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaba70e-feb2-4c96-b3e0-c118d86b4f5c None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:11.715094 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 118/222] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:54:11 2022] GET /identity/v3/users/0e523b449ff64dc59804790956bb6524 => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:11.721665 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaba70e-feb2-4c96-b3e0-c118d86b4f5c None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:11.779998 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:11.799112 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:11.847914 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f625e5e7-45ba-491f-bd79-5e4856974620 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:11.863313 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaba70e-feb2-4c96-b3e0-c118d86b4f5c None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:11.896421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05f4af4-3447-4e44-8590-58503cebb77c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:11.897086 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f4af4-3447-4e44-8590-58503cebb77c None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:11.897538 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f4af4-3447-4e44-8590-58503cebb77c None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:11.897648 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05f4af4-3447-4e44-8590-58503cebb77c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:11.912338 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e785d-383e-4c2c-8212-46a990c5ad60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:11.913212 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e785d-383e-4c2c-8212-46a990c5ad60 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:11.913498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e785d-383e-4c2c-8212-46a990c5ad60 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:11.913778 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e785d-383e-4c2c-8212-46a990c5ad60 None admin] PATH_INFO: `/v3/projects/aa8007020ec543c1a6405283ddd21067` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:11.915826 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84e785d-383e-4c2c-8212-46a990c5ad60 None admin] RBAC: Authorizing `identity:get_project(project_id=aa8007020ec543c1a6405283ddd21067)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:11.917825 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84e785d-383e-4c2c-8212-46a990c5ad60 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:11.919806 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 119/223] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:54:11 2022] GET /identity/v3/projects/aa8007020ec543c1a6405283ddd21067 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:11.920798 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f1f46-8265-44da-b127-d7e820bf8ff6 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:11.921087 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f1f46-8265-44da-b127-d7e820bf8ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:11.921366 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162f1f46-8265-44da-b127-d7e820bf8ff6 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:11.922725 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 120/224] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:11.927288 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f1f46-8265-44da-b127-d7e820bf8ff6 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:11.993624 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05f4af4-3447-4e44-8590-58503cebb77c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:11.996099 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05f4af4-3447-4e44-8590-58503cebb77c None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:12.036221 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162f1f46-8265-44da-b127-d7e820bf8ff6 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:12.045601 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:12.073133 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1550a186-9da5-41c0-90e7-4a13ac31f240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:12.074028 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1550a186-9da5-41c0-90e7-4a13ac31f240 None admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:12.074359 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1550a186-9da5-41c0-90e7-4a13ac31f240 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:12.074583 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1550a186-9da5-41c0-90e7-4a13ac31f240 None admin] PATH_INFO: `/v3/projects/aa8007020ec543c1a6405283ddd21067/users/0e523b449ff64dc59804790956bb6524/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:12.077594 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1550a186-9da5-41c0-90e7-4a13ac31f240 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa8007020ec543c1a6405283ddd21067, user_id=0e523b449ff64dc59804790956bb6524, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:12.079088 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1550a186-9da5-41c0-90e7-4a13ac31f240 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:12.088121 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:12.120606 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 105/225] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:54:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 06 16:54:12.126698 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9456a-cb6f-4f8e-8e2b-0e7789ffde0e None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:12.126986 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9456a-cb6f-4f8e-8e2b-0e7789ffde0e None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:12.127267 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b9456a-cb6f-4f8e-8e2b-0e7789ffde0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:12.145324 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-82b9456a-cb6f-4f8e-8e2b-0e7789ffde0e None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:12.146761 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-82b9456a-cb6f-4f8e-8e2b-0e7789ffde0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:12.147449 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 121/226] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 6 16:54:11 2022] PUT /identity/v3/projects/aa8007020ec543c1a6405283ddd21067/users/0e523b449ff64dc59804790956bb6524/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 16:54:12.148773 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1550a186-9da5-41c0-90e7-4a13ac31f240 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:12.243755 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 106/227] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:12.275884 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b9456a-cb6f-4f8e-8e2b-0e7789ffde0e None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:12.305586 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1550a186-9da5-41c0-90e7-4a13ac31f240 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:12.350454 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0178245-3cdb-4e1c-8a5e-1c6aeeeacac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:12.351333 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0178245-3cdb-4e1c-8a5e-1c6aeeeacac0 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:12.351618 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0178245-3cdb-4e1c-8a5e-1c6aeeeacac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:12.351933 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0178245-3cdb-4e1c-8a5e-1c6aeeeacac0 None admin] PATH_INFO: `/v3/projects/aa8007020ec543c1a6405283ddd21067` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:12.353159 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0178245-3cdb-4e1c-8a5e-1c6aeeeacac0 None admin] RBAC: Authorizing `identity:get_project(project_id=aa8007020ec543c1a6405283ddd21067)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:12.354786 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0178245-3cdb-4e1c-8a5e-1c6aeeeacac0 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:12.356645 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 122/228] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:54:12 2022] GET /identity/v3/projects/aa8007020ec543c1a6405283ddd21067 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:12.364685 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0178245-3cdb-4e1c-8a5e-1c6aeeeacac0 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:12.400083 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:12.420594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b9456a-cb6f-4f8e-8e2b-0e7789ffde0e None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:12.423502 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:12.459219 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6acb9cb-c152-4138-ae3c-d0c521050401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:12.460084 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acb9cb-c152-4138-ae3c-d0c521050401 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:12.460373 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acb9cb-c152-4138-ae3c-d0c521050401 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:12.460660 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6acb9cb-c152-4138-ae3c-d0c521050401 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:12.476021 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6acb9cb-c152-4138-ae3c-d0c521050401 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:12.482521 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0178245-3cdb-4e1c-8a5e-1c6aeeeacac0 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:12.492805 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-b6acb9cb-c152-4138-ae3c-d0c521050401 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 16:54:12.492805 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:12.492805 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:12.492805 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:12.492805 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:12.492805 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:12.492805 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:12.492805 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:12.492805 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:12.492805 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:12.492805 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:12.492805 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:12.492805 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:12.492805 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:12.492805 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:12.492805 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:12.493765 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:12.493765 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:12.493765 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:12.493765 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:12.493765 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:12.493765 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:12.493765 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:12.493765 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:12.493765 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:12.493765 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:12.493765 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:12.493765 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:12.493765 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:12.493765 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:12.493765 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:12.493765 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:12.494540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:12.494540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:12.494540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:12.494540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:12.494540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:12.494540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:12.494540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:12.494540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:12.494540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:12.494540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:12.494540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 16:54:12.494540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:12.494540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 107/229] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 6 16:54:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:12.500446 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6acb9cb-c152-4138-ae3c-d0c521050401 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:12.544786 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c5c71-b0b7-40ea-b680-8c00ce713524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:12.545836 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c5c71-b0b7-40ea-b680-8c00ce713524 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:12.546319 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c5c71-b0b7-40ea-b680-8c00ce713524 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:12.546790 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c5c71-b0b7-40ea-b680-8c00ce713524 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:12.547950 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2c5c71-b0b7-40ea-b680-8c00ce713524 None admin] RBAC: Adding query filter params (role.id=7fd63d2d99254c00bb3501173643ee30, scope.project.id=aa8007020ec543c1a6405283ddd21067, user.id=2ad8193e60d54439a0c591bb264dff53) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:12.548454 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2c5c71-b0b7-40ea-b680-8c00ce713524 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:12.552652 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2c5c71-b0b7-40ea-b680-8c00ce713524 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:12.561257 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 123/230] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:54:12 2022] GET /identity/v3/role_assignments?user.id=2ad8193e60d54439a0c591bb264dff53&scope.project.id=aa8007020ec543c1a6405283ddd21067&role.id=7fd63d2d99254c00bb3501173643ee30 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:12.687771 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6acb9cb-c152-4138-ae3c-d0c521050401 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:12.744519 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8eb542-eed9-4e87-829c-c6fbf3489243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:12.745247 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8eb542-eed9-4e87-829c-c6fbf3489243 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:12.745534 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8eb542-eed9-4e87-829c-c6fbf3489243 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:12.745812 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8eb542-eed9-4e87-829c-c6fbf3489243 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:12.746582 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8eb542-eed9-4e87-829c-c6fbf3489243 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:12.746892 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8eb542-eed9-4e87-829c-c6fbf3489243 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:12.748331 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8eb542-eed9-4e87-829c-c6fbf3489243 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:12.757685 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 108/231] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:54:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:12.763326 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c5c71-b0b7-40ea-b680-8c00ce713524 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:12.871336 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:12.880121 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c5c71-b0b7-40ea-b680-8c00ce713524 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:12.929361 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d693699-3051-48a5-89e3-c0f842be278c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:12.930272 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d693699-3051-48a5-89e3-c0f842be278c None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:12.930558 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d693699-3051-48a5-89e3-c0f842be278c None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:12.930839 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d693699-3051-48a5-89e3-c0f842be278c None admin] PATH_INFO: `/v3/users/0e523b449ff64dc59804790956bb6524` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:12.932227 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d693699-3051-48a5-89e3-c0f842be278c None admin] RBAC: Authorizing `identity:get_user(user_id=0e523b449ff64dc59804790956bb6524)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:12.933723 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d693699-3051-48a5-89e3-c0f842be278c None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:12.935394 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 124/232] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:54:12 2022] GET /identity/v3/users/0e523b449ff64dc59804790956bb6524 => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:12.941048 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8eb542-eed9-4e87-829c-c6fbf3489243 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:13.002592 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:13.110102 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8eb542-eed9-4e87-829c-c6fbf3489243 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:13.164432 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09311a3f-b83c-4aa1-b6bc-644cdaf4abe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:13.165325 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09311a3f-b83c-4aa1-b6bc-644cdaf4abe5 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:13.165610 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09311a3f-b83c-4aa1-b6bc-644cdaf4abe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:13.165893 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09311a3f-b83c-4aa1-b6bc-644cdaf4abe5 None admin] PATH_INFO: `/v3/projects/aa8007020ec543c1a6405283ddd21067` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:13.167173 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09311a3f-b83c-4aa1-b6bc-644cdaf4abe5 None admin] RBAC: Authorizing `identity:get_project(project_id=aa8007020ec543c1a6405283ddd21067)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:13.168660 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09311a3f-b83c-4aa1-b6bc-644cdaf4abe5 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:13.170682 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 109/233] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:54:12 2022] GET /identity/v3/projects/aa8007020ec543c1a6405283ddd21067 => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:13.175955 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d693699-3051-48a5-89e3-c0f842be278c None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:13.315270 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d693699-3051-48a5-89e3-c0f842be278c None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:13.352589 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62e9bf6-472c-4808-b008-e158e5418cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:13.353486 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e9bf6-472c-4808-b008-e158e5418cbe None admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:13.353765 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e9bf6-472c-4808-b008-e158e5418cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:13.354035 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e9bf6-472c-4808-b008-e158e5418cbe None admin] PATH_INFO: `/v3/projects/aa8007020ec543c1a6405283ddd21067/users/0e523b449ff64dc59804790956bb6524/roles/c31c5ee7ed8646e48fa81e85d10fcaf2` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:13.356398 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62e9bf6-472c-4808-b008-e158e5418cbe None admin] RBAC: Authorizing `identity:create_grant(project_id=aa8007020ec543c1a6405283ddd21067, user_id=0e523b449ff64dc59804790956bb6524, role_id=c31c5ee7ed8646e48fa81e85d10fcaf2)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:13.358061 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62e9bf6-472c-4808-b008-e158e5418cbe None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:13.448918 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 125/234] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 6 16:54:13 2022] PUT /identity/v3/projects/aa8007020ec543c1a6405283ddd21067/users/0e523b449ff64dc59804790956bb6524/roles/c31c5ee7ed8646e48fa81e85d10fcaf2 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 16:54:14.052532 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f8887-5215-4676-99cc-10a9c4435127 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:14.053021 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f8887-5215-4676-99cc-10a9c4435127 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:14.053021 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f8887-5215-4676-99cc-10a9c4435127 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:14.054698 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 110/235] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:14.060371 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e433d4-0f89-4657-9091-ad5010846e6b None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:14.060872 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e433d4-0f89-4657-9091-ad5010846e6b None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:14.060945 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e433d4-0f89-4657-9091-ad5010846e6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:14.084174 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-c3e433d4-0f89-4657-9091-ad5010846e6b None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:14.085381 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c3e433d4-0f89-4657-9091-ad5010846e6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:14.107833 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:14.155905 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 126/236] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:14.169943 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234a73f-e59c-4c9d-bd2a-14d8211fa148 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:14.170514 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234a73f-e59c-4c9d-bd2a-14d8211fa148 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:14.170994 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a234a73f-e59c-4c9d-bd2a-14d8211fa148 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:14.172716 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 111/237] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:14.178083 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdfe615-4683-480a-b64e-2597b87a79e0 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:14.178603 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdfe615-4683-480a-b64e-2597b87a79e0 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:14.179095 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdfe615-4683-480a-b64e-2597b87a79e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:14.196781 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-8cdfe615-4683-480a-b64e-2597b87a79e0 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:14.198339 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-8cdfe615-4683-480a-b64e-2597b87a79e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:14.202516 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a234a73f-e59c-4c9d-bd2a-14d8211fa148 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:14.325978 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 127/238] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:14.361436 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:14.370897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a234a73f-e59c-4c9d-bd2a-14d8211fa148 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:14.380105 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdfe615-4683-480a-b64e-2597b87a79e0 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:14.440102 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829dbfd7-70e6-47fc-b9f3-000f34cf3b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:14.440911 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829dbfd7-70e6-47fc-b9f3-000f34cf3b4e None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:14.441194 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829dbfd7-70e6-47fc-b9f3-000f34cf3b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:14.441527 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829dbfd7-70e6-47fc-b9f3-000f34cf3b4e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:14.444299 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829dbfd7-70e6-47fc-b9f3-000f34cf3b4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:14.445757 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829dbfd7-70e6-47fc-b9f3-000f34cf3b4e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:14.448615 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 112/239] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:54:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:14.453680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829dbfd7-70e6-47fc-b9f3-000f34cf3b4e None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:14.533149 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdfe615-4683-480a-b64e-2597b87a79e0 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:14.536051 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:14.597725 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a504bfc7-ac82-45ed-81a1-a1ce4332e552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:14.598874 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504bfc7-ac82-45ed-81a1-a1ce4332e552 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:14.599363 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504bfc7-ac82-45ed-81a1-a1ce4332e552 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:14.599865 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504bfc7-ac82-45ed-81a1-a1ce4332e552 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:14.620091 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a504bfc7-ac82-45ed-81a1-a1ce4332e552 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:14.628217 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-a504bfc7-ac82-45ed-81a1-a1ce4332e552 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 16:54:14.628217 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:14.628217 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:14.628217 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:14.628217 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:14.628217 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:14.628217 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:14.628217 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:14.628217 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:14.628217 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:14.628217 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:14.628217 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:14.628217 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:14.628217 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:14.628217 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:14.628217 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:14.629081 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:14.629081 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:14.629081 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:14.629081 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:14.629081 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:14.629081 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:14.629081 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:14.629081 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:14.629081 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:14.629081 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:14.629081 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:14.629081 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:14.629081 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:14.629081 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:14.629081 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:14.629081 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:14.629860 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:14.629860 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:14.629860 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:14.629860 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:14.629860 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:14.629860 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:14.629860 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:14.629860 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:14.629860 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:14.629860 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:14.629860 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 16:54:14.629860 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:14.638736 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 128/240] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:54:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:14.642668 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a504bfc7-ac82-45ed-81a1-a1ce4332e552 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:14.645899 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829dbfd7-70e6-47fc-b9f3-000f34cf3b4e None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:14.690231 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc4b97b-bc62-4523-847f-5d34e80ae6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:14.691269 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc4b97b-bc62-4523-847f-5d34e80ae6ee None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:14.691779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc4b97b-bc62-4523-847f-5d34e80ae6ee None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:14.692254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc4b97b-bc62-4523-847f-5d34e80ae6ee None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:14.693196 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc4b97b-bc62-4523-847f-5d34e80ae6ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:14.694957 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc4b97b-bc62-4523-847f-5d34e80ae6ee None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:14.788226 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a504bfc7-ac82-45ed-81a1-a1ce4332e552 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:14.824546 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25734c36-e023-4bb6-a26c-63fa5e167918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:14.825264 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25734c36-e023-4bb6-a26c-63fa5e167918 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:14.825564 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25734c36-e023-4bb6-a26c-63fa5e167918 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:14.825913 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25734c36-e023-4bb6-a26c-63fa5e167918 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:14.826668 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25734c36-e023-4bb6-a26c-63fa5e167918 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:14.826978 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25734c36-e023-4bb6-a26c-63fa5e167918 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:14.828420 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25734c36-e023-4bb6-a26c-63fa5e167918 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:14.836925 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 129/241] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 6 16:54:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:14.841380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd7bae-7264-4702-a5ef-843b1f96c016 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:14.841380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd7bae-7264-4702-a5ef-843b1f96c016 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:14.841380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cd7bae-7264-4702-a5ef-843b1f96c016 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:14.841380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 130/242] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:14.841942 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cd7bae-7264-4702-a5ef-843b1f96c016 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:14.863320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:14.863483 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 113/243] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 6 16:54:14 2022] POST /identity/v3/users => generated 293 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 16:54:14.863483 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec3059-8ce0-4bf7-9b46-90f6b0dd4d23 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:14.863631 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec3059-8ce0-4bf7-9b46-90f6b0dd4d23 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:14.863948 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec3059-8ce0-4bf7-9b46-90f6b0dd4d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:14.883165 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-f0ec3059-8ce0-4bf7-9b46-90f6b0dd4d23 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:14.884486 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-f0ec3059-8ce0-4bf7-9b46-90f6b0dd4d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:14.896693 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:14.962876 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 114/244] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:14.966413 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cd7bae-7264-4702-a5ef-843b1f96c016 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:15.007689 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ec3059-8ce0-4bf7-9b46-90f6b0dd4d23 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:15.022279 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550b275d-0ea4-4849-a730-a5c0af2ef45e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:15.023443 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b275d-0ea4-4849-a730-a5c0af2ef45e None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:15.023756 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b275d-0ea4-4849-a730-a5c0af2ef45e None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:15.024052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550b275d-0ea4-4849-a730-a5c0af2ef45e None admin] PATH_INFO: `/v3/users/753f662c626846f5bb6592f912deb949` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:15.042086 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550b275d-0ea4-4849-a730-a5c0af2ef45e None admin] RBAC: Authorizing `identity:get_user(user_id=753f662c626846f5bb6592f912deb949)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:15.043577 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550b275d-0ea4-4849-a730-a5c0af2ef45e None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:15.045820 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 131/245] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:54:14 2022] GET /identity/v3/users/753f662c626846f5bb6592f912deb949 => generated 297 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:15.051113 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550b275d-0ea4-4849-a730-a5c0af2ef45e None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:15.139142 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:15.165287 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ec3059-8ce0-4bf7-9b46-90f6b0dd4d23 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:15.185238 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550b275d-0ea4-4849-a730-a5c0af2ef45e None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:15.217068 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81de124-e6d0-44b9-b74f-d78eff5ae4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:15.220758 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81de124-e6d0-44b9-b74f-d78eff5ae4ed None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:15.220758 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81de124-e6d0-44b9-b74f-d78eff5ae4ed None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:15.220758 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81de124-e6d0-44b9-b74f-d78eff5ae4ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:15.232534 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81de124-e6d0-44b9-b74f-d78eff5ae4ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:15.232765 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d038a58-0941-4d80-affc-27c0ade21259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:15.233449 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d038a58-0941-4d80-affc-27c0ade21259 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:15.233740 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d038a58-0941-4d80-affc-27c0ade21259 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:15.234015 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d038a58-0941-4d80-affc-27c0ade21259 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:15.240670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-c81de124-e6d0-44b9-b74f-d78eff5ae4ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:54:15.240670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:15.240670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:15.240670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:15.240670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:15.240670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:15.240670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:15.240670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:15.240670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:15.240670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:15.240670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:15.240670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:15.240670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:15.240670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:15.240670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:15.240670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:15.241560 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:15.241560 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:15.241560 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:15.241560 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:15.241560 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:15.241560 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:15.241560 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:15.241560 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:15.241560 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:15.241560 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:15.241560 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:15.241560 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:15.241560 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:15.241560 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:15.241560 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:15.241560 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:15.242396 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:15.242396 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:15.242396 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:15.242396 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:15.242396 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:15.242396 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:15.242396 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:15.242396 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:15.242396 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:15.242396 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:15.242396 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:54:15.242396 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:15.242396 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 115/246] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 6 16:54:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:54:15.243506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c7eee1-64dd-4b9f-ad77-1100f7bb91d6 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:15.244212 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c7eee1-64dd-4b9f-ad77-1100f7bb91d6 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:15.244212 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c7eee1-64dd-4b9f-ad77-1100f7bb91d6 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:15.246073 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 116/247] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:15.251011 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c7eee1-64dd-4b9f-ad77-1100f7bb91d6 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:15.254724 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:15.333332 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d038a58-0941-4d80-affc-27c0ade21259 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:15.334832 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d038a58-0941-4d80-affc-27c0ade21259 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:15.403043 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c7eee1-64dd-4b9f-ad77-1100f7bb91d6 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:15.410491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:15.447450 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572c7e14-f22c-4164-bdb8-dc1cd958ecf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:15.448580 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c7e14-f22c-4164-bdb8-dc1cd958ecf2 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:15.449122 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c7e14-f22c-4164-bdb8-dc1cd958ecf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:15.449603 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c7e14-f22c-4164-bdb8-dc1cd958ecf2 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:15.450078 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 132/248] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:54:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 16:54:15.450795 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572c7e14-f22c-4164-bdb8-dc1cd958ecf2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:15.451288 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572c7e14-f22c-4164-bdb8-dc1cd958ecf2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:15.451741 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ce119-fc6b-4d54-bb02-ea9e1f710fd6 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:15.452389 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ce119-fc6b-4d54-bb02-ea9e1f710fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:15.452991 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259ce119-fc6b-4d54-bb02-ea9e1f710fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:15.454037 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572c7e14-f22c-4164-bdb8-dc1cd958ecf2 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:15.464884 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 117/249] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:54:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:15.466108 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572c7e14-f22c-4164-bdb8-dc1cd958ecf2 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:15.471517 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-259ce119-fc6b-4d54-bb02-ea9e1f710fd6 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:15.472749 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-259ce119-fc6b-4d54-bb02-ea9e1f710fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:15.540742 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:15.548003 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 133/250] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:15.548197 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ce119-fc6b-4d54-bb02-ea9e1f710fd6 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:15.582457 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572c7e14-f22c-4164-bdb8-dc1cd958ecf2 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:15.621471 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d514188a-e306-403e-93ee-c0b12ed926a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:15.622788 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514188a-e306-403e-93ee-c0b12ed926a3 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:15.623302 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514188a-e306-403e-93ee-c0b12ed926a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:15.623874 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d514188a-e306-403e-93ee-c0b12ed926a3 None admin] PATH_INFO: `/v3/projects/aa8007020ec543c1a6405283ddd21067` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:15.625289 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d514188a-e306-403e-93ee-c0b12ed926a3 None admin] RBAC: Authorizing `identity:get_project(project_id=aa8007020ec543c1a6405283ddd21067)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:15.626971 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d514188a-e306-403e-93ee-c0b12ed926a3 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:15.630030 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 118/251] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:54:15 2022] GET /identity/v3/projects/aa8007020ec543c1a6405283ddd21067 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:15.631578 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d514188a-e306-403e-93ee-c0b12ed926a3 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:15.649165 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:15.655615 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259ce119-fc6b-4d54-bb02-ea9e1f710fd6 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:15.691613 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c63fe2-7efe-4cd2-a66b-004e95299fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:15.692736 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c63fe2-7efe-4cd2-a66b-004e95299fb2 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:15.693021 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c63fe2-7efe-4cd2-a66b-004e95299fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:15.693299 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c63fe2-7efe-4cd2-a66b-004e95299fb2 None admin] PATH_INFO: `/v3/users/0e523b449ff64dc59804790956bb6524` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:15.694519 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c63fe2-7efe-4cd2-a66b-004e95299fb2 None admin] RBAC: Authorizing `identity:get_user(user_id=0e523b449ff64dc59804790956bb6524)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:15.696471 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c63fe2-7efe-4cd2-a66b-004e95299fb2 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:15.698239 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 134/252] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:54:15 2022] GET /identity/v3/users/0e523b449ff64dc59804790956bb6524 => generated 283 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:15.699083 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c63fe2-7efe-4cd2-a66b-004e95299fb2 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:15.753167 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d514188a-e306-403e-93ee-c0b12ed926a3 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:15.804955 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c63fe2-7efe-4cd2-a66b-004e95299fb2 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:15.816952 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82648129-3c74-4b95-a0b3-29a31f7fa5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:15.818011 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82648129-3c74-4b95-a0b3-29a31f7fa5d5 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:15.818541 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82648129-3c74-4b95-a0b3-29a31f7fa5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:15.819041 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82648129-3c74-4b95-a0b3-29a31f7fa5d5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:15.821156 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:15.835788 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82648129-3c74-4b95-a0b3-29a31f7fa5d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:15.847521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-82648129-3c74-4b95-a0b3-29a31f7fa5d5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 16:54:15.847521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:15.847521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:15.847521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:15.847521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:15.847521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:15.847521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:15.847521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:15.847521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:15.847521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:15.847521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:15.847521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:15.847521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:15.847521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:15.847521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:15.847521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:15.848701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:15.848701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:15.848701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:15.848701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:15.848701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:15.848701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:15.848701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:15.848701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:15.848701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:15.848701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:15.848701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:15.848701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:15.848701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:15.848701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:15.848701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:15.848701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:15.849521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:15.849521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:15.849521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:15.849521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:15.849521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:15.849521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:15.849521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:15.849521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:15.849521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:15.849521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:15.849521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 16:54:15.849521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:15.851424 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a47bf-82f5-4371-85f7-eb3ed3e2a2ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:15.852272 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a47bf-82f5-4371-85f7-eb3ed3e2a2ab None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:15.852694 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a47bf-82f5-4371-85f7-eb3ed3e2a2ab None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:15.853112 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a47bf-82f5-4371-85f7-eb3ed3e2a2ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:15.854167 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74a47bf-82f5-4371-85f7-eb3ed3e2a2ab None admin] RBAC: Adding query filter params (role.id=e9f69ea47fce47fab94435de5a29de5f, scope.project.id=aa8007020ec543c1a6405283ddd21067, user.id=753f662c626846f5bb6592f912deb949) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:15.855197 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74a47bf-82f5-4371-85f7-eb3ed3e2a2ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:15.856973 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74a47bf-82f5-4371-85f7-eb3ed3e2a2ab None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:15.861076 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 119/253] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:54:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:15.862397 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82648129-3c74-4b95-a0b3-29a31f7fa5d5 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:15.869973 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 135/254] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:54:15 2022] GET /identity/v3/role_assignments?user.id=753f662c626846f5bb6592f912deb949&scope.project.id=aa8007020ec543c1a6405283ddd21067&role.id=e9f69ea47fce47fab94435de5a29de5f => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:15.871253 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a47bf-82f5-4371-85f7-eb3ed3e2a2ab None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:15.920292 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:16.002345 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a47bf-82f5-4371-85f7-eb3ed3e2a2ab None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:16.057921 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82648129-3c74-4b95-a0b3-29a31f7fa5d5 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:16.075139 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4338b49-f1a0-4138-92ed-6dbae49e30a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:16.076147 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4338b49-f1a0-4138-92ed-6dbae49e30a7 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:16.076788 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4338b49-f1a0-4138-92ed-6dbae49e30a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:16.078307 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4338b49-f1a0-4138-92ed-6dbae49e30a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:16.079532 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4338b49-f1a0-4138-92ed-6dbae49e30a7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:16.080034 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4338b49-f1a0-4138-92ed-6dbae49e30a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:16.082340 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4338b49-f1a0-4138-92ed-6dbae49e30a7 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:16.101097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 136/255] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 6 16:54:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:16.114314 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4338b49-f1a0-4138-92ed-6dbae49e30a7 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:16.129668 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090b9f8b-1831-429f-ab45-5abe5ec4d700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:16.130347 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b9f8b-1831-429f-ab45-5abe5ec4d700 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:16.130762 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b9f8b-1831-429f-ab45-5abe5ec4d700 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:16.131124 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b9f8b-1831-429f-ab45-5abe5ec4d700 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:16.202955 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:16.210463 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090b9f8b-1831-429f-ab45-5abe5ec4d700 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:16.212103 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090b9f8b-1831-429f-ab45-5abe5ec4d700 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:16.232333 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4338b49-f1a0-4138-92ed-6dbae49e30a7 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:16.236089 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:16.275176 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc6e04d-673d-423e-92ce-3ced1e964a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:16.276414 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6e04d-673d-423e-92ce-3ced1e964a46 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:16.276846 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6e04d-673d-423e-92ce-3ced1e964a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:16.277311 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6e04d-673d-423e-92ce-3ced1e964a46 None admin] PATH_INFO: `/v3/users/2ad8193e60d54439a0c591bb264dff53` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:16.278678 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc6e04d-673d-423e-92ce-3ced1e964a46 None admin] RBAC: Authorizing `identity:get_user(user_id=2ad8193e60d54439a0c591bb264dff53)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:16.280345 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc6e04d-673d-423e-92ce-3ced1e964a46 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:16.282469 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 137/256] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:54:16 2022] GET /identity/v3/users/2ad8193e60d54439a0c591bb264dff53 => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:16.283972 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da705a43-df19-4fa2-957d-1678f131ea05 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:16.284446 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da705a43-df19-4fa2-957d-1678f131ea05 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:16.284918 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da705a43-df19-4fa2-957d-1678f131ea05 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:16.286323 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 138/257] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:16.291796 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b875a61-3552-4c36-b108-f67e8b800328 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:16.292305 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b875a61-3552-4c36-b108-f67e8b800328 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:16.292791 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b875a61-3552-4c36-b108-f67e8b800328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:16.361084 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-0b875a61-3552-4c36-b108-f67e8b800328 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:16.363567 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-0b875a61-3552-4c36-b108-f67e8b800328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:16.454965 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 139/258] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:16.455352 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b875a61-3552-4c36-b108-f67e8b800328 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:16.487979 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 120/259] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:54:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 626 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 16:54:16.493643 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090b9f8b-1831-429f-ab45-5abe5ec4d700 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:16.612863 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:16.629217 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090b9f8b-1831-429f-ab45-5abe5ec4d700 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:16.662375 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:16.676917 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b875a61-3552-4c36-b108-f67e8b800328 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:16.704080 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9fc7ef-98a6-49aa-afd1-daf178c82947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:16.704957 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9fc7ef-98a6-49aa-afd1-daf178c82947 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:16.705301 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9fc7ef-98a6-49aa-afd1-daf178c82947 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:16.705526 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9fc7ef-98a6-49aa-afd1-daf178c82947 None admin] PATH_INFO: `/v3/projects/aa8007020ec543c1a6405283ddd21067` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:16.708758 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9fc7ef-98a6-49aa-afd1-daf178c82947 None admin] RBAC: Authorizing `identity:get_project(project_id=aa8007020ec543c1a6405283ddd21067)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:16.710207 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9fc7ef-98a6-49aa-afd1-daf178c82947 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:16.718237 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 121/260] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:54:16 2022] GET /identity/v3/projects/aa8007020ec543c1a6405283ddd21067 => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:16.718870 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9fc7ef-98a6-49aa-afd1-daf178c82947 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:16.736524 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7944ab-3989-4d2a-b2a0-5ae422fd83df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:16.737416 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7944ab-3989-4d2a-b2a0-5ae422fd83df None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:16.737710 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7944ab-3989-4d2a-b2a0-5ae422fd83df None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:16.737994 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7944ab-3989-4d2a-b2a0-5ae422fd83df None admin] PATH_INFO: `/v3/projects/aa8007020ec543c1a6405283ddd21067` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:16.739207 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7944ab-3989-4d2a-b2a0-5ae422fd83df None admin] RBAC: Authorizing `identity:get_project(project_id=aa8007020ec543c1a6405283ddd21067)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:16.740804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7944ab-3989-4d2a-b2a0-5ae422fd83df None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:16.742653 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 140/261] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:54:16 2022] GET /identity/v3/projects/aa8007020ec543c1a6405283ddd21067 => generated 302 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:16.743430 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7944ab-3989-4d2a-b2a0-5ae422fd83df None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:16.852206 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7944ab-3989-4d2a-b2a0-5ae422fd83df None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:16.890899 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8482d749-2b3e-451f-8cde-08a63e86fe9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:16.891570 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8482d749-2b3e-451f-8cde-08a63e86fe9d None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:16.891906 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8482d749-2b3e-451f-8cde-08a63e86fe9d None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:16.892193 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8482d749-2b3e-451f-8cde-08a63e86fe9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:16.893296 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8482d749-2b3e-451f-8cde-08a63e86fe9d None admin] RBAC: Adding query filter params (role.id=b41f8a7a58e64f749f2099ca45cc04a7, scope.project.id=aa8007020ec543c1a6405283ddd21067, user.id=0e523b449ff64dc59804790956bb6524) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:16.893717 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8482d749-2b3e-451f-8cde-08a63e86fe9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:16.895267 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8482d749-2b3e-451f-8cde-08a63e86fe9d None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:16.902298 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 141/262] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:54:16 2022] GET /identity/v3/role_assignments?user.id=0e523b449ff64dc59804790956bb6524&scope.project.id=aa8007020ec543c1a6405283ddd21067&role.id=b41f8a7a58e64f749f2099ca45cc04a7 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:16.903191 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8482d749-2b3e-451f-8cde-08a63e86fe9d None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:16.932235 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:17.014337 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8482d749-2b3e-451f-8cde-08a63e86fe9d None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:17.037763 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9fc7ef-98a6-49aa-afd1-daf178c82947 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:17.058007 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d26c74-32a1-4aad-ba04-a6518e66364a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:17.059276 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d26c74-32a1-4aad-ba04-a6518e66364a None admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:17.059819 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d26c74-32a1-4aad-ba04-a6518e66364a None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:17.060172 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d26c74-32a1-4aad-ba04-a6518e66364a None admin] PATH_INFO: `/v3/projects/aa8007020ec543c1a6405283ddd21067/users/2ad8193e60d54439a0c591bb264dff53/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:17.063066 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d26c74-32a1-4aad-ba04-a6518e66364a None admin] RBAC: Authorizing `identity:create_grant(project_id=aa8007020ec543c1a6405283ddd21067, user_id=2ad8193e60d54439a0c591bb264dff53, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:17.064897 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d26c74-32a1-4aad-ba04-a6518e66364a None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:17.130964 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d1eb11-4311-41fb-b66e-9fb95e531388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:17.131811 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d1eb11-4311-41fb-b66e-9fb95e531388 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:17.132100 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d1eb11-4311-41fb-b66e-9fb95e531388 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:17.132374 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d1eb11-4311-41fb-b66e-9fb95e531388 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:17.152199 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d1eb11-4311-41fb-b66e-9fb95e531388 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:17.161000 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 142/263] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 6 16:54:16 2022] PUT /identity/v3/projects/aa8007020ec543c1a6405283ddd21067/users/2ad8193e60d54439a0c591bb264dff53/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 16:54:17.164204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-71d1eb11-4311-41fb-b66e-9fb95e531388 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 16:54:17.164204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:17.164204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:17.164204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:17.164204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:17.164204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:17.164204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:17.164204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:17.164204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:17.164204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:17.164204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:17.164204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:17.164204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:17.164204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:17.164204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:17.164204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:17.165281 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:17.165281 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:17.165281 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:17.165281 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:17.165281 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:17.165281 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:17.165281 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:17.165281 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:17.165281 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:17.165281 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:17.165281 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:17.165281 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:17.165281 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:17.165281 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:17.165281 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:17.165281 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:17.166285 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:17.166285 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:17.166285 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:17.166285 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:17.166285 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:17.166285 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:17.166285 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:17.166285 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:17.166285 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:17.166285 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:17.166285 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 16:54:17.166285 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:17.169050 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 122/264] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 6 16:54:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 451 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:17.177068 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d26c74-32a1-4aad-ba04-a6518e66364a None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:17.300241 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:17.318555 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d26c74-32a1-4aad-ba04-a6518e66364a None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:17.358067 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff74c5b-dff5-4b59-90cc-fdd8f463428f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:17.359200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff74c5b-dff5-4b59-90cc-fdd8f463428f None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:17.359498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff74c5b-dff5-4b59-90cc-fdd8f463428f None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:17.359891 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff74c5b-dff5-4b59-90cc-fdd8f463428f None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:17.361406 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff74c5b-dff5-4b59-90cc-fdd8f463428f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:17.361970 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff74c5b-dff5-4b59-90cc-fdd8f463428f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:17.363565 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff74c5b-dff5-4b59-90cc-fdd8f463428f None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:17.374479 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 143/265] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:54:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:17.379861 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d1eb11-4311-41fb-b66e-9fb95e531388 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:17.408597 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:17.495759 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d1eb11-4311-41fb-b66e-9fb95e531388 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:17.520938 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679346a9-9c92-486e-86fe-68b1d56dde6a None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:17.521501 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679346a9-9c92-486e-86fe-68b1d56dde6a None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:17.521773 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679346a9-9c92-486e-86fe-68b1d56dde6a None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:17.524218 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 144/266] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:17.531081 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c55bb-20a5-4699-8b61-139e2ed3808a None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:17.531376 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c55bb-20a5-4699-8b61-139e2ed3808a None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:17.531657 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405c55bb-20a5-4699-8b61-139e2ed3808a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:17.549064 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856b4e7-cccb-4e0d-8796-e901eb6b0074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:17.549953 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856b4e7-cccb-4e0d-8796-e901eb6b0074 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:17.550305 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856b4e7-cccb-4e0d-8796-e901eb6b0074 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:17.550520 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856b4e7-cccb-4e0d-8796-e901eb6b0074 None admin] PATH_INFO: `/v3/users/0e523b449ff64dc59804790956bb6524` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:17.551125 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-405c55bb-20a5-4699-8b61-139e2ed3808a None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:17.552430 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9856b4e7-cccb-4e0d-8796-e901eb6b0074 None admin] RBAC: Authorizing `identity:get_user(user_id=0e523b449ff64dc59804790956bb6524)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:17.553893 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9856b4e7-cccb-4e0d-8796-e901eb6b0074 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:17.554289 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-405c55bb-20a5-4699-8b61-139e2ed3808a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:17.556701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 123/267] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:54:17 2022] GET /identity/v3/users/0e523b449ff64dc59804790956bb6524 => generated 283 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:17.561320 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856b4e7-cccb-4e0d-8796-e901eb6b0074 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:17.638381 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 145/268] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 06 16:54:17.666593 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c55bb-20a5-4699-8b61-139e2ed3808a None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:17.674481 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:17.685577 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856b4e7-cccb-4e0d-8796-e901eb6b0074 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:17.722037 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6659912f-08f6-49c4-aee2-6103ace763d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:17.722696 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659912f-08f6-49c4-aee2-6103ace763d8 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:17.722984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659912f-08f6-49c4-aee2-6103ace763d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:17.723368 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6659912f-08f6-49c4-aee2-6103ace763d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:17.750574 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:17.790487 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405c55bb-20a5-4699-8b61-139e2ed3808a None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:17.799853 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6659912f-08f6-49c4-aee2-6103ace763d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:17.801743 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6659912f-08f6-49c4-aee2-6103ace763d8 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:17.846037 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f9aed1-5a48-4d4a-86e0-898277ddbb8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:17.847130 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9aed1-5a48-4d4a-86e0-898277ddbb8a None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:17.847570 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9aed1-5a48-4d4a-86e0-898277ddbb8a None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:17.849118 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9aed1-5a48-4d4a-86e0-898277ddbb8a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:17.876369 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f9aed1-5a48-4d4a-86e0-898277ddbb8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:17.892268 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-34f9aed1-5a48-4d4a-86e0-898277ddbb8a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 16:54:17.892268 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:17.892268 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:17.892268 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:17.892268 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:17.892268 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:17.892268 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:17.892268 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:17.892268 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:17.892268 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:17.892268 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:17.892268 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:17.892268 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:17.892268 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:17.892268 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:17.892268 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:17.893193 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:17.893193 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:17.893193 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:17.893193 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:17.893193 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:17.893193 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:17.893193 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:17.893193 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:17.893193 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:17.893193 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:17.893193 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:17.893193 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:17.893193 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:17.893193 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:17.893193 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:17.893193 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:17.893959 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:17.893959 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:17.893959 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:17.893959 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:17.893959 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:17.893959 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:17.893959 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:17.893959 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:17.893959 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:17.893959 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:17.893959 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 16:54:17.893959 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:17.893959 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 146/269] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:54:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:17.914187 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f9aed1-5a48-4d4a-86e0-898277ddbb8a None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:17.952101 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:17.961114 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 124/270] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:54:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 16:54:17.967048 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6659912f-08f6-49c4-aee2-6103ace763d8 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:18.093642 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f9aed1-5a48-4d4a-86e0-898277ddbb8a None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:18.110355 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6659912f-08f6-49c4-aee2-6103ace763d8 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:18.126350 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:18.131498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb383b5-01e1-408d-9bed-b508b8bfe579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:18.132229 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb383b5-01e1-408d-9bed-b508b8bfe579 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:18.132517 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb383b5-01e1-408d-9bed-b508b8bfe579 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:18.132797 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb383b5-01e1-408d-9bed-b508b8bfe579 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:18.133567 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb383b5-01e1-408d-9bed-b508b8bfe579 None admin] RBAC: Adding query filter params (name=member) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:18.133975 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb383b5-01e1-408d-9bed-b508b8bfe579 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:18.136006 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb383b5-01e1-408d-9bed-b508b8bfe579 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:18.146694 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 147/271] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 6 16:54:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:18.148112 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb26203-43cc-4111-8c6c-ed7c24625a93 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:18.148403 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb26203-43cc-4111-8c6c-ed7c24625a93 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:18.148681 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb26203-43cc-4111-8c6c-ed7c24625a93 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:18.149945 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 148/272] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:18.152621 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb26203-43cc-4111-8c6c-ed7c24625a93 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:18.153955 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1893c12f-2d2e-42c6-88ec-ddbb53efdae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:18.155418 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893c12f-2d2e-42c6-88ec-ddbb53efdae4 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:18.155709 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893c12f-2d2e-42c6-88ec-ddbb53efdae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:18.156395 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893c12f-2d2e-42c6-88ec-ddbb53efdae4 None admin] PATH_INFO: `/v3/projects/aa8007020ec543c1a6405283ddd21067` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:18.157613 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1893c12f-2d2e-42c6-88ec-ddbb53efdae4 None admin] RBAC: Authorizing `identity:get_project(project_id=aa8007020ec543c1a6405283ddd21067)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:18.159372 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1893c12f-2d2e-42c6-88ec-ddbb53efdae4 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:18.167087 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 125/273] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:54:17 2022] GET /identity/v3/projects/aa8007020ec543c1a6405283ddd21067 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:18.167087 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9489d3-62e8-444d-b974-9e553c656382 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:18.167087 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9489d3-62e8-444d-b974-9e553c656382 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:18.167087 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9489d3-62e8-444d-b974-9e553c656382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:18.186009 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-7c9489d3-62e8-444d-b974-9e553c656382 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:18.187358 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-7c9489d3-62e8-444d-b974-9e553c656382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:18.247672 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:18.264154 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 126/274] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:18.265234 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9489d3-62e8-444d-b974-9e553c656382 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:18.297872 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb26203-43cc-4111-8c6c-ed7c24625a93 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:18.339091 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1f92b-4cf3-41d8-be4e-36f7bdcfdfeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:18.340111 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1f92b-4cf3-41d8-be4e-36f7bdcfdfeb None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:18.340469 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1f92b-4cf3-41d8-be4e-36f7bdcfdfeb None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:18.340816 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1f92b-4cf3-41d8-be4e-36f7bdcfdfeb None admin] PATH_INFO: `/v3/users/8dfcf3bc25ef4136aba694857496a897` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:18.355797 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e1f92b-4cf3-41d8-be4e-36f7bdcfdfeb None admin] RBAC: Authorizing `identity:get_user(user_id=8dfcf3bc25ef4136aba694857496a897)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:18.357286 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e1f92b-4cf3-41d8-be4e-36f7bdcfdfeb None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:18.359450 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 149/275] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:54:18 2022] GET /identity/v3/users/8dfcf3bc25ef4136aba694857496a897 => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:18.360401 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1f92b-4cf3-41d8-be4e-36f7bdcfdfeb None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:18.392179 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9489d3-62e8-444d-b974-9e553c656382 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:18.425265 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:18.435623 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36d19e9-4076-42dd-a9b6-3ff5ac46339a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:18.438079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d19e9-4076-42dd-a9b6-3ff5ac46339a None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:18.438079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d19e9-4076-42dd-a9b6-3ff5ac46339a None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:18.438079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d19e9-4076-42dd-a9b6-3ff5ac46339a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:18.438079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36d19e9-4076-42dd-a9b6-3ff5ac46339a None admin] RBAC: Adding query filter params (role.id=c31c5ee7ed8646e48fa81e85d10fcaf2, scope.project.id=aa8007020ec543c1a6405283ddd21067, user.id=0e523b449ff64dc59804790956bb6524) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:18.438452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36d19e9-4076-42dd-a9b6-3ff5ac46339a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:18.440024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36d19e9-4076-42dd-a9b6-3ff5ac46339a None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:18.448693 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 127/276] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:54:18 2022] GET /identity/v3/role_assignments?user.id=0e523b449ff64dc59804790956bb6524&scope.project.id=aa8007020ec543c1a6405283ddd21067&role.id=c31c5ee7ed8646e48fa81e85d10fcaf2 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:18.452558 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36d19e9-4076-42dd-a9b6-3ff5ac46339a None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:18.507932 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e1f92b-4cf3-41d8-be4e-36f7bdcfdfeb None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:18.557613 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:18.568157 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fb6641-cfd9-46f3-b4ca-f5c2ff817580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:18.569660 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb6641-cfd9-46f3-b4ca-f5c2ff817580 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:18.570326 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb6641-cfd9-46f3-b4ca-f5c2ff817580 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:18.571157 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb6641-cfd9-46f3-b4ca-f5c2ff817580 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:18.573648 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fb6641-cfd9-46f3-b4ca-f5c2ff817580 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:18.575581 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fb6641-cfd9-46f3-b4ca-f5c2ff817580 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:18.578542 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 150/277] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:54:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:18.584929 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031acb4-67a8-4995-85e6-649a84f98dc1 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:18.585613 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031acb4-67a8-4995-85e6-649a84f98dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:18.586125 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031acb4-67a8-4995-85e6-649a84f98dc1 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:18.587829 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 151/278] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:18 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:18.588879 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3031acb4-67a8-4995-85e6-649a84f98dc1 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:18.629608 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36d19e9-4076-42dd-a9b6-3ff5ac46339a None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:18.676977 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19198f67-cf23-4ef8-9ffa-a22d7271b18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:18.678160 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19198f67-cf23-4ef8-9ffa-a22d7271b18d None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:18.678448 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19198f67-cf23-4ef8-9ffa-a22d7271b18d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:18.678773 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19198f67-cf23-4ef8-9ffa-a22d7271b18d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:18.679623 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19198f67-cf23-4ef8-9ffa-a22d7271b18d None admin] RBAC: Adding query filter params (role.id=7fd63d2d99254c00bb3501173643ee30, user.id=8dfcf3bc25ef4136aba694857496a897, scope.system=all) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:18.680012 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19198f67-cf23-4ef8-9ffa-a22d7271b18d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:18.681496 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19198f67-cf23-4ef8-9ffa-a22d7271b18d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:18.687632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 128/279] 10.222.0.40 () {56 vars in 1308 bytes} [Wed Jul 6 16:54:18 2022] GET /identity/v3/role_assignments?user.id=8dfcf3bc25ef4136aba694857496a897&scope.system=all&role.id=7fd63d2d99254c00bb3501173643ee30 => generated 226 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:18.688998 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c77543-9247-4193-b1a0-b0405a951908 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:18.689549 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c77543-9247-4193-b1a0-b0405a951908 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:18.689549 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c77543-9247-4193-b1a0-b0405a951908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:18.709380 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-e5c77543-9247-4193-b1a0-b0405a951908 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:18.710864 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e5c77543-9247-4193-b1a0-b0405a951908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:18.724935 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3031acb4-67a8-4995-85e6-649a84f98dc1 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:18.755141 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:18.775050 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee2827a-292e-4290-94c1-79b7a2f70f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:18.775805 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2827a-292e-4290-94c1-79b7a2f70f95 None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:18.783426 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2827a-292e-4290-94c1-79b7a2f70f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:18.783426 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2827a-292e-4290-94c1-79b7a2f70f95 None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:18.783426 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee2827a-292e-4290-94c1-79b7a2f70f95 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:18.783426 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee2827a-292e-4290-94c1-79b7a2f70f95 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:18.815588 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 129/280] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:18.840539 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c77543-9247-4193-b1a0-b0405a951908 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:18.864703 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 152/281] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 6 16:54:18 2022] POST /identity/v3/users => generated 293 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 16:54:18.898247 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:18.959539 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c77543-9247-4193-b1a0-b0405a951908 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:19.002680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890c7e0-b333-4a31-94b0-83cbf6c252fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:19.003912 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890c7e0-b333-4a31-94b0-83cbf6c252fc None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:19.004433 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890c7e0-b333-4a31-94b0-83cbf6c252fc None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:19.004913 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890c7e0-b333-4a31-94b0-83cbf6c252fc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:19.016571 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2890c7e0-b333-4a31-94b0-83cbf6c252fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:19.025013 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-2890c7e0-b333-4a31-94b0-83cbf6c252fc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 16:54:19.025013 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:19.025013 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:19.025013 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:19.025013 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:19.025013 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:19.025013 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:19.025013 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:19.025013 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:19.025013 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:19.025013 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:19.025013 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:19.025013 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:19.025013 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:19.025013 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:19.025013 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:19.025910 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:19.025910 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:19.025910 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:19.025910 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:19.025910 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:19.025910 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:19.025910 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:19.025910 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:19.025910 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:19.025910 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:19.025910 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:19.025910 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:19.025910 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:19.025910 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:19.025910 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:19.025910 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:19.026731 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:19.026731 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:19.026731 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:19.026731 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:19.026731 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:19.026731 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:19.026731 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:19.026731 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:19.026731 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:19.026731 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:19.026731 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 16:54:19.026731 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:19.028788 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 130/282] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:54:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:54:19.034698 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee2827a-292e-4290-94c1-79b7a2f70f95 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:19.140449 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee2827a-292e-4290-94c1-79b7a2f70f95 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:19.177145 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def05471-3211-43cb-bdbf-2f6973aab8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:19.178156 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def05471-3211-43cb-bdbf-2f6973aab8dc None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:19.178601 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def05471-3211-43cb-bdbf-2f6973aab8dc None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:19.179023 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def05471-3211-43cb-bdbf-2f6973aab8dc None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:19.179959 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def05471-3211-43cb-bdbf-2f6973aab8dc None admin] RBAC: Adding query filter params (name=reader) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:19.180419 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def05471-3211-43cb-bdbf-2f6973aab8dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:19.181957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def05471-3211-43cb-bdbf-2f6973aab8dc None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:19.191392 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 153/283] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 6 16:54:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:19.197153 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890c7e0-b333-4a31-94b0-83cbf6c252fc None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:19.321701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890c7e0-b333-4a31-94b0-83cbf6c252fc None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:19.330003 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:19.358673 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73d60e-f81b-4d2a-978e-abc45307b1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:19.359617 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73d60e-f81b-4d2a-978e-abc45307b1fe None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:19.359939 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73d60e-f81b-4d2a-978e-abc45307b1fe None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:19.360217 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f73d60e-f81b-4d2a-978e-abc45307b1fe None admin] PATH_INFO: `/v3/users/753f662c626846f5bb6592f912deb949` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:19.361457 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f73d60e-f81b-4d2a-978e-abc45307b1fe None admin] RBAC: Authorizing `identity:get_user(user_id=753f662c626846f5bb6592f912deb949)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:19.362914 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f73d60e-f81b-4d2a-978e-abc45307b1fe None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:19.364900 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 131/284] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:54:19 2022] GET /identity/v3/users/753f662c626846f5bb6592f912deb949 => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:19.371378 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def05471-3211-43cb-bdbf-2f6973aab8dc None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:19.423684 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:19.503269 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def05471-3211-43cb-bdbf-2f6973aab8dc None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:19.549046 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a371310-2cde-4227-ab4d-0f29acf068d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:19.549929 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a371310-2cde-4227-ab4d-0f29acf068d0 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:19.550215 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a371310-2cde-4227-ab4d-0f29acf068d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:19.550496 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a371310-2cde-4227-ab4d-0f29acf068d0 None admin] PATH_INFO: `/v3/projects/aa8007020ec543c1a6405283ddd21067` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:19.551703 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a371310-2cde-4227-ab4d-0f29acf068d0 None admin] RBAC: Authorizing `identity:get_project(project_id=aa8007020ec543c1a6405283ddd21067)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:19.553246 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a371310-2cde-4227-ab4d-0f29acf068d0 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:19.555180 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 154/285] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:54:19 2022] GET /identity/v3/projects/aa8007020ec543c1a6405283ddd21067 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:19.560715 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73d60e-f81b-4d2a-978e-abc45307b1fe None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:19.701218 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f73d60e-f81b-4d2a-978e-abc45307b1fe None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:19.748665 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce9544-25ae-4d92-a992-62f2cbf02ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:19.749571 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce9544-25ae-4d92-a992-62f2cbf02ea3 None admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:19.749852 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce9544-25ae-4d92-a992-62f2cbf02ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:19.750124 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce9544-25ae-4d92-a992-62f2cbf02ea3 None admin] PATH_INFO: `/v3/projects/aa8007020ec543c1a6405283ddd21067/users/753f662c626846f5bb6592f912deb949/roles/e9f69ea47fce47fab94435de5a29de5f` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:19.757185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce9544-25ae-4d92-a992-62f2cbf02ea3 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa8007020ec543c1a6405283ddd21067, user_id=753f662c626846f5bb6592f912deb949, role_id=e9f69ea47fce47fab94435de5a29de5f)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:19.758584 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce9544-25ae-4d92-a992-62f2cbf02ea3 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:19.783263 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30035fbe-244f-4bc4-9626-085d688e2e62 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:19.783553 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30035fbe-244f-4bc4-9626-085d688e2e62 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:19.783865 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30035fbe-244f-4bc4-9626-085d688e2e62 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:19.785246 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 155/286] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 16:54:19.790043 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d602b90-73e8-4d1d-8a52-8bca8bfdaa63 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:19.790335 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d602b90-73e8-4d1d-8a52-8bca8bfdaa63 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:19.790618 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d602b90-73e8-4d1d-8a52-8bca8bfdaa63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:19.807386 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-0d602b90-73e8-4d1d-8a52-8bca8bfdaa63 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:19.808623 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-0d602b90-73e8-4d1d-8a52-8bca8bfdaa63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:19.835365 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 132/287] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 6 16:54:19 2022] PUT /identity/v3/projects/aa8007020ec543c1a6405283ddd21067/users/753f662c626846f5bb6592f912deb949/roles/e9f69ea47fce47fab94435de5a29de5f => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 16:54:19.900261 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 156/288] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 16:54:19.924592 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce9544-25ae-4d92-a992-62f2cbf02ea3 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:19.964796 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:20.047415 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce9544-25ae-4d92-a992-62f2cbf02ea3 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:20.085480 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dc8684-bc74-471c-bc9b-ab628a06b94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:20.086283 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc8684-bc74-471c-bc9b-ab628a06b94e None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:20.086566 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc8684-bc74-471c-bc9b-ab628a06b94e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:20.086842 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dc8684-bc74-471c-bc9b-ab628a06b94e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:20.097205 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dc8684-bc74-471c-bc9b-ab628a06b94e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:20.105132 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-57dc8684-bc74-471c-bc9b-ab628a06b94e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 16:54:20.105132 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:20.105132 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:20.105132 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:20.105132 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:20.105132 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:20.105132 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:20.105132 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:20.105132 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:20.105132 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:20.105132 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:20.105132 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:20.105132 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:20.105132 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:20.105132 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:20.105132 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:20.105984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:20.105984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:20.105984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:20.105984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:20.105984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:20.105984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:20.105984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:20.105984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:20.105984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:20.105984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:20.105984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:20.105984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:20.105984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:20.105984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:20.105984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:20.105984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:20.106756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:20.106756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:20.106756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:20.106756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:20.106756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:20.106756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:20.106756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:20.106756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:20.106756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:20.106756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:20.106756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 16:54:20.106756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:20.113707 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d602b90-73e8-4d1d-8a52-8bca8bfdaa63 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:20.115764 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 133/289] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:54:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:20.161361 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:20.247615 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d602b90-73e8-4d1d-8a52-8bca8bfdaa63 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:20.297324 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0b1d77-d663-46b0-a2e7-db203c175ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:20.298070 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b1d77-d663-46b0-a2e7-db203c175ab8 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:20.298305 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b1d77-d663-46b0-a2e7-db203c175ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:20.298632 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b1d77-d663-46b0-a2e7-db203c175ab8 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:20.299398 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0b1d77-d663-46b0-a2e7-db203c175ab8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:20.299759 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0b1d77-d663-46b0-a2e7-db203c175ab8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:20.301153 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0b1d77-d663-46b0-a2e7-db203c175ab8 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:20.311330 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 157/290] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 6 16:54:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:20.316773 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dc8684-bc74-471c-bc9b-ab628a06b94e None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:20.441148 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:20.448761 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dc8684-bc74-471c-bc9b-ab628a06b94e None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:20.487703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e595a9-2711-46a1-b952-88edf5b6203f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:20.488648 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e595a9-2711-46a1-b952-88edf5b6203f None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:20.488938 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e595a9-2711-46a1-b952-88edf5b6203f None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:20.489218 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e595a9-2711-46a1-b952-88edf5b6203f None admin] PATH_INFO: `/v3/users/2ad8193e60d54439a0c591bb264dff53` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:20.490484 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e595a9-2711-46a1-b952-88edf5b6203f None admin] RBAC: Authorizing `identity:get_user(user_id=2ad8193e60d54439a0c591bb264dff53)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:20.492021 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e595a9-2711-46a1-b952-88edf5b6203f None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:20.494126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 134/291] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:54:20 2022] GET /identity/v3/users/2ad8193e60d54439a0c591bb264dff53 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:20.499444 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0b1d77-d663-46b0-a2e7-db203c175ab8 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:20.630633 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:20.639422 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0b1d77-d663-46b0-a2e7-db203c175ab8 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:20.683334 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaad9ec-a1a4-4437-8e42-20367a895913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:20.684037 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaad9ec-a1a4-4437-8e42-20367a895913 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:20.684323 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaad9ec-a1a4-4437-8e42-20367a895913 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:20.684594 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaad9ec-a1a4-4437-8e42-20367a895913 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:20.767398 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaad9ec-a1a4-4437-8e42-20367a895913 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:20.768935 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaad9ec-a1a4-4437-8e42-20367a895913 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:20.900016 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:20.904570 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 158/292] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:54:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 16:54:20.910215 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e595a9-2711-46a1-b952-88edf5b6203f None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:21.056465 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e595a9-2711-46a1-b952-88edf5b6203f None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:21.097586 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1d705c-413b-484d-83e4-8c8058f13cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:21.098828 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d705c-413b-484d-83e4-8c8058f13cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:21.099316 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d705c-413b-484d-83e4-8c8058f13cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:21.099840 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1d705c-413b-484d-83e4-8c8058f13cb1 None admin] PATH_INFO: `/v3/projects/aa8007020ec543c1a6405283ddd21067` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:21.101237 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1d705c-413b-484d-83e4-8c8058f13cb1 None admin] RBAC: Authorizing `identity:get_project(project_id=aa8007020ec543c1a6405283ddd21067)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:21.102916 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1d705c-413b-484d-83e4-8c8058f13cb1 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:21.105472 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 135/293] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:54:20 2022] GET /identity/v3/projects/aa8007020ec543c1a6405283ddd21067 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:21.111348 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaad9ec-a1a4-4437-8e42-20367a895913 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:21.250427 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaad9ec-a1a4-4437-8e42-20367a895913 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:21.291197 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d46e554-5bc3-4f67-8eaa-a7d1b84309f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:21.292423 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46e554-5bc3-4f67-8eaa-a7d1b84309f4 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:21.292709 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46e554-5bc3-4f67-8eaa-a7d1b84309f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:21.292989 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d46e554-5bc3-4f67-8eaa-a7d1b84309f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:21.293845 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d46e554-5bc3-4f67-8eaa-a7d1b84309f4 None admin] RBAC: Adding query filter params (role.id=7fd63d2d99254c00bb3501173643ee30, scope.project.id=aa8007020ec543c1a6405283ddd21067, user.id=2ad8193e60d54439a0c591bb264dff53) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:21.294165 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d46e554-5bc3-4f67-8eaa-a7d1b84309f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:21.295608 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d46e554-5bc3-4f67-8eaa-a7d1b84309f4 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:21.301942 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 159/294] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:54:21 2022] GET /identity/v3/role_assignments?user.id=2ad8193e60d54439a0c591bb264dff53&scope.project.id=aa8007020ec543c1a6405283ddd21067&role.id=7fd63d2d99254c00bb3501173643ee30 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:21.316971 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e85ba-9cb6-4465-b5a1-33fd97196ebc None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:21.317298 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e85ba-9cb6-4465-b5a1-33fd97196ebc None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:21.317545 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e85ba-9cb6-4465-b5a1-33fd97196ebc None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:21.318976 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 136/295] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:21.325456 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a930ba3b-9f8e-4cb2-a4ca-1fe49fdce3e3 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:21.325827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a930ba3b-9f8e-4cb2-a4ca-1fe49fdce3e3 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:21.326116 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a930ba3b-9f8e-4cb2-a4ca-1fe49fdce3e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:21.344419 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-a930ba3b-9f8e-4cb2-a4ca-1fe49fdce3e3 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:21.345643 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-a930ba3b-9f8e-4cb2-a4ca-1fe49fdce3e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:21.414899 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:21.427667 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 160/296] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 16:54:21.451673 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e85ba-9cb6-4465-b5a1-33fd97196ebc None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:21.484388 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:21.562778 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e85ba-9cb6-4465-b5a1-33fd97196ebc None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:21.598640 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851b49e-41b4-4393-bbfc-5420ea1fc58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:21.599446 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851b49e-41b4-4393-bbfc-5420ea1fc58c None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:21.599746 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851b49e-41b4-4393-bbfc-5420ea1fc58c None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:21.600033 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851b49e-41b4-4393-bbfc-5420ea1fc58c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:21.611124 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b851b49e-41b4-4393-bbfc-5420ea1fc58c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:21.618940 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-b851b49e-41b4-4393-bbfc-5420ea1fc58c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 16:54:21.618940 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:21.618940 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:21.618940 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:21.618940 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:21.618940 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:21.618940 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:21.618940 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:21.618940 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:21.618940 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:21.618940 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:21.618940 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:21.618940 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:21.618940 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:21.618940 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:21.618940 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:21.619850 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:21.619850 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:21.619850 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:21.619850 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:21.619850 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:21.619850 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:21.619850 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:21.619850 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:21.619850 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:21.619850 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:21.619850 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:21.619850 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:21.619850 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:21.619850 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:21.619850 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:21.619850 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:21.620653 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:21.620653 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:21.620653 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:21.620653 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:21.620653 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:21.620653 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:21.620653 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:21.620653 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:21.620653 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:21.620653 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:21.620653 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 16:54:21.620653 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:21.620653 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 137/297] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:54:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:21.626418 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a930ba3b-9f8e-4cb2-a4ca-1fe49fdce3e3 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:21.698475 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef3af22-b19b-4cad-957e-7a69b38ab9ee None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:21.698776 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef3af22-b19b-4cad-957e-7a69b38ab9ee None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:21.699038 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef3af22-b19b-4cad-957e-7a69b38ab9ee None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:21.700710 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 138/298] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 16:54:21.706494 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d901079-76f4-4bb6-aa6a-ce4c77f5a4fe None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:21.707100 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d901079-76f4-4bb6-aa6a-ce4c77f5a4fe None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:21.707576 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d901079-76f4-4bb6-aa6a-ce4c77f5a4fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:21.732356 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-2d901079-76f4-4bb6-aa6a-ce4c77f5a4fe None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:21.734532 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-2d901079-76f4-4bb6-aa6a-ce4c77f5a4fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:21.749332 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a930ba3b-9f8e-4cb2-a4ca-1fe49fdce3e3 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:21.797018 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9159176-ad19-42b7-9b3b-40d52e2cae67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:21.797714 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9159176-ad19-42b7-9b3b-40d52e2cae67 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:21.798000 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9159176-ad19-42b7-9b3b-40d52e2cae67 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:21.798277 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9159176-ad19-42b7-9b3b-40d52e2cae67 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:21.799036 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9159176-ad19-42b7-9b3b-40d52e2cae67 None admin] RBAC: Adding query filter params (name=member) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:21.799346 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9159176-ad19-42b7-9b3b-40d52e2cae67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:21.801378 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9159176-ad19-42b7-9b3b-40d52e2cae67 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:21.813598 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 161/299] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 6 16:54:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:21.818047 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 139/300] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 16:54:21.823919 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9159176-ad19-42b7-9b3b-40d52e2cae67 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:21.825571 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a83fab-8cb7-49ae-824e-ff28b4cb3b85 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:21.825859 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a83fab-8cb7-49ae-824e-ff28b4cb3b85 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:21.826147 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a83fab-8cb7-49ae-824e-ff28b4cb3b85 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:21.828726 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 140/301] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:21.832755 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f2aab-f803-4098-9c6a-de4cc96582c0 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:21.833303 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f2aab-f803-4098-9c6a-de4cc96582c0 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:21.833780 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91f2aab-f803-4098-9c6a-de4cc96582c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:21.857289 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-d91f2aab-f803-4098-9c6a-de4cc96582c0 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:21.859021 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d91f2aab-f803-4098-9c6a-de4cc96582c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:21.875491 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:21.882634 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:21.926937 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9159176-ad19-42b7-9b3b-40d52e2cae67 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:21.931946 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 141/302] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 16:54:21.931946 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f2aab-f803-4098-9c6a-de4cc96582c0 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:21.965426 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86230e1e-a9db-4d34-bd92-b6ae429506a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:21.966307 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86230e1e-a9db-4d34-bd92-b6ae429506a6 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:21.966587 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86230e1e-a9db-4d34-bd92-b6ae429506a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:21.966857 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86230e1e-a9db-4d34-bd92-b6ae429506a6 None admin] PATH_INFO: `/v3/users/8dfcf3bc25ef4136aba694857496a897` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:21.968614 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86230e1e-a9db-4d34-bd92-b6ae429506a6 None admin] RBAC: Authorizing `identity:get_user(user_id=8dfcf3bc25ef4136aba694857496a897)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:21.970052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86230e1e-a9db-4d34-bd92-b6ae429506a6 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:21.976284 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 162/303] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:54:21 2022] GET /identity/v3/users/8dfcf3bc25ef4136aba694857496a897 => generated 293 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:21.976284 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86230e1e-a9db-4d34-bd92-b6ae429506a6 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:22.040197 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91f2aab-f803-4098-9c6a-de4cc96582c0 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:22.076000 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52d7d28-8c5f-48a6-81cd-b1c0f1aa7047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:22.076720 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52d7d28-8c5f-48a6-81cd-b1c0f1aa7047 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:22.077001 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52d7d28-8c5f-48a6-81cd-b1c0f1aa7047 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:22.077275 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52d7d28-8c5f-48a6-81cd-b1c0f1aa7047 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:22.078064 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86230e1e-a9db-4d34-bd92-b6ae429506a6 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:22.078429 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52d7d28-8c5f-48a6-81cd-b1c0f1aa7047 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:22.080042 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52d7d28-8c5f-48a6-81cd-b1c0f1aa7047 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:22.081934 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 142/304] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:54:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:22.084589 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52d7d28-8c5f-48a6-81cd-b1c0f1aa7047 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:22.104530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:22.104713 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:22.116273 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c6e46b-ebc4-4fb8-86c4-72ef208726a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:22.117062 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6e46b-ebc4-4fb8-86c4-72ef208726a6 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:22.119327 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6e46b-ebc4-4fb8-86c4-72ef208726a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:22.119327 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c6e46b-ebc4-4fb8-86c4-72ef208726a6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:22.128780 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c6e46b-ebc4-4fb8-86c4-72ef208726a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:22.136526 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-b2c6e46b-ebc4-4fb8-86c4-72ef208726a6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 16:54:22.136526 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:22.136526 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:22.136526 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:22.136526 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:22.136526 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:22.136526 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:22.136526 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:22.136526 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:22.136526 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:22.136526 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:22.136526 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:22.136526 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:22.136526 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:22.136526 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:22.136526 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:22.137425 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:22.137425 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:22.137425 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:22.137425 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:22.137425 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:22.137425 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:22.137425 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:22.137425 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:22.137425 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:22.137425 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:22.137425 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:22.137425 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:22.137425 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:22.137425 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:22.137425 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:22.137425 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:22.138204 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:22.138204 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:22.138204 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:22.138204 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:22.138204 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:22.138204 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:22.138204 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:22.138204 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:22.138204 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:22.138204 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:22.138204 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 16:54:22.138204 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:22.138204 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 163/305] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:54:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:22.138851 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c6e46b-ebc4-4fb8-86c4-72ef208726a6 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:22.191957 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52d7d28-8c5f-48a6-81cd-b1c0f1aa7047 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:22.225655 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c24cdc-d5e8-422e-896f-b9719f5a16a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:22.226515 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c24cdc-d5e8-422e-896f-b9719f5a16a8 None admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:22.228071 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c24cdc-d5e8-422e-896f-b9719f5a16a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:22.228071 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c24cdc-d5e8-422e-896f-b9719f5a16a8 None admin] PATH_INFO: `/v3/system/users/8dfcf3bc25ef4136aba694857496a897/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:22.228587 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c24cdc-d5e8-422e-896f-b9719f5a16a8 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=8dfcf3bc25ef4136aba694857496a897, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:22.230495 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c24cdc-d5e8-422e-896f-b9719f5a16a8 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:22.244057 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c6e46b-ebc4-4fb8-86c4-72ef208726a6 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:22.280352 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a83c5f-9718-4e78-9eea-59f64567103c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:22.281035 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a83c5f-9718-4e78-9eea-59f64567103c None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:22.281323 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a83c5f-9718-4e78-9eea-59f64567103c None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:22.281605 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a83c5f-9718-4e78-9eea-59f64567103c None admin] PATH_INFO: `/v3/groups` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:22.282302 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a83c5f-9718-4e78-9eea-59f64567103c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:22.283770 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a83c5f-9718-4e78-9eea-59f64567103c None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:22.385576 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 143/306] 10.222.0.40 () {58 vars in 1334 bytes} [Wed Jul 6 16:54:22 2022] PUT /identity/v3/system/users/8dfcf3bc25ef4136aba694857496a897/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 16:54:22.386474 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c24cdc-d5e8-422e-896f-b9719f5a16a8 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:22.431229 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 164/307] 10.222.0.40 () {60 vars in 1134 bytes} [Wed Jul 6 16:54:22 2022] POST /identity/v3/groups => generated 229 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 16:54:22.432645 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d18080-e78a-401d-80f2-f8001fce7be7 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:22.432935 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d18080-e78a-401d-80f2-f8001fce7be7 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:22.433214 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d18080-e78a-401d-80f2-f8001fce7be7 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:22.434712 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 165/308] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:22.440692 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72993b6-b212-4851-b963-a0aa85695109 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:22.441015 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72993b6-b212-4851-b963-a0aa85695109 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:22.441260 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72993b6-b212-4851-b963-a0aa85695109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:22.461072 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-c72993b6-b212-4851-b963-a0aa85695109 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:22.462357 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c72993b6-b212-4851-b963-a0aa85695109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:22.498252 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:22.512186 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:22.533466 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c24cdc-d5e8-422e-896f-b9719f5a16a8 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:22.555145 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 166/309] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 16:54:22.580664 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72993b6-b212-4851-b963-a0aa85695109 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:22.586206 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a7a3fb-17ee-4e41-bd89-3a011951c508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:22.586206 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a7a3fb-17ee-4e41-bd89-3a011951c508 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:22.586206 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a7a3fb-17ee-4e41-bd89-3a011951c508 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:22.586206 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a7a3fb-17ee-4e41-bd89-3a011951c508 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:22.586560 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a7a3fb-17ee-4e41-bd89-3a011951c508 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:22.586560 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a7a3fb-17ee-4e41-bd89-3a011951c508 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:22.586850 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a7a3fb-17ee-4e41-bd89-3a011951c508 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:22.599027 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 144/310] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 6 16:54:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:22.605247 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a7a3fb-17ee-4e41-bd89-3a011951c508 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:22.709183 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72993b6-b212-4851-b963-a0aa85695109 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:22.736673 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a7a3fb-17ee-4e41-bd89-3a011951c508 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:22.766605 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd86d48a-5141-4f32-b07a-3c2326f7cbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:22.768432 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86d48a-5141-4f32-b07a-3c2326f7cbfb None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:22.768920 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86d48a-5141-4f32-b07a-3c2326f7cbfb None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:22.769441 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86d48a-5141-4f32-b07a-3c2326f7cbfb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:22.772330 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96cc8cb-485a-4c35-b11b-daf3ebfb4bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:22.773316 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96cc8cb-485a-4c35-b11b-daf3ebfb4bbc None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:22.773651 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96cc8cb-485a-4c35-b11b-daf3ebfb4bbc None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:22.773997 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96cc8cb-485a-4c35-b11b-daf3ebfb4bbc None admin] PATH_INFO: `/v3/users/2f07f64cea16495d9eea180957631259` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:22.784824 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd86d48a-5141-4f32-b07a-3c2326f7cbfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:22.789214 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96cc8cb-485a-4c35-b11b-daf3ebfb4bbc None admin] RBAC: Authorizing `identity:get_user(user_id=2f07f64cea16495d9eea180957631259)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:22.790810 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96cc8cb-485a-4c35-b11b-daf3ebfb4bbc None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:22.792963 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 145/311] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:54:22 2022] GET /identity/v3/users/2f07f64cea16495d9eea180957631259 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:22.794714 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-cd86d48a-5141-4f32-b07a-3c2326f7cbfb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 16:54:22.794714 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:22.794714 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:22.794714 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:22.794714 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:22.794714 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:22.794714 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:22.794714 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:22.794714 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:22.794714 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:22.794714 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:22.794714 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:22.794714 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:22.794714 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:22.794714 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:22.794714 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:22.795572 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:22.795572 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:22.795572 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:22.795572 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:22.795572 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:22.795572 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:22.795572 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:22.795572 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:22.795572 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:22.795572 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:22.795572 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:22.795572 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:22.795572 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:22.795572 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:22.795572 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:22.795572 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:22.796380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:22.796380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:22.796380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:22.796380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:22.796380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:22.796380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:22.796380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:22.796380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:22.796380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:22.796380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:22.796380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 16:54:22.796380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:22.798572 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96cc8cb-485a-4c35-b11b-daf3ebfb4bbc None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:22.798780 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 167/312] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:54:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:54:22.803941 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:22.806134 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd86d48a-5141-4f32-b07a-3c2326f7cbfb None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:22.847285 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:22.917241 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96cc8cb-485a-4c35-b11b-daf3ebfb4bbc None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:22.936552 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd86d48a-5141-4f32-b07a-3c2326f7cbfb None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:22.961756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4949e056-c2a1-452e-b2e6-c82b9f186342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:22.962439 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4949e056-c2a1-452e-b2e6-c82b9f186342 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:22.962854 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4949e056-c2a1-452e-b2e6-c82b9f186342 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:22.963775 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4949e056-c2a1-452e-b2e6-c82b9f186342 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:22.965300 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4949e056-c2a1-452e-b2e6-c82b9f186342 None admin] RBAC: Adding query filter params (role.id=e9f69ea47fce47fab94435de5a29de5f, user.id=2f07f64cea16495d9eea180957631259, scope.system=all) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:22.965707 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4949e056-c2a1-452e-b2e6-c82b9f186342 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:22.967487 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4949e056-c2a1-452e-b2e6-c82b9f186342 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:22.975842 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 146/313] 10.222.0.40 () {56 vars in 1308 bytes} [Wed Jul 6 16:54:22 2022] GET /identity/v3/role_assignments?user.id=2f07f64cea16495d9eea180957631259&scope.system=all&role.id=e9f69ea47fce47fab94435de5a29de5f => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:22.991300 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b77624-cbf1-44a1-bd89-20b5a9578ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:22.992062 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b77624-cbf1-44a1-bd89-20b5a9578ef0 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:22.992315 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b77624-cbf1-44a1-bd89-20b5a9578ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:22.992590 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b77624-cbf1-44a1-bd89-20b5a9578ef0 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:22.993369 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b77624-cbf1-44a1-bd89-20b5a9578ef0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:22.993678 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b77624-cbf1-44a1-bd89-20b5a9578ef0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:22.995786 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b77624-cbf1-44a1-bd89-20b5a9578ef0 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:23.006530 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 168/314] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 6 16:54:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:23.013006 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4949e056-c2a1-452e-b2e6-c82b9f186342 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:23.092136 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:23.136950 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4949e056-c2a1-452e-b2e6-c82b9f186342 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:23.172188 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e34aa0-7027-4309-8e25-5f88f39f9d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:23.173080 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e34aa0-7027-4309-8e25-5f88f39f9d23 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:23.173367 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e34aa0-7027-4309-8e25-5f88f39f9d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:23.173646 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e34aa0-7027-4309-8e25-5f88f39f9d23 None admin] PATH_INFO: `/v3/users/753f662c626846f5bb6592f912deb949` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:23.174915 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e34aa0-7027-4309-8e25-5f88f39f9d23 None admin] RBAC: Authorizing `identity:get_user(user_id=753f662c626846f5bb6592f912deb949)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:23.176451 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e34aa0-7027-4309-8e25-5f88f39f9d23 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:23.178413 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 147/315] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:54:23 2022] GET /identity/v3/users/753f662c626846f5bb6592f912deb949 => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:23.184338 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b77624-cbf1-44a1-bd89-20b5a9578ef0 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:23.280861 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:23.287010 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b77624-cbf1-44a1-bd89-20b5a9578ef0 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:23.321010 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cba74d-841f-4cf0-9d30-0711c3334ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:23.321662 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cba74d-841f-4cf0-9d30-0711c3334ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:23.322003 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cba74d-841f-4cf0-9d30-0711c3334ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:23.322278 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2cba74d-841f-4cf0-9d30-0711c3334ee4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:23.397685 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cba74d-841f-4cf0-9d30-0711c3334ee4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:23.399179 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2cba74d-841f-4cf0-9d30-0711c3334ee4 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:23.499282 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:23.506785 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 169/316] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:54:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 16:54:23.512417 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e34aa0-7027-4309-8e25-5f88f39f9d23 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:23.625689 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e34aa0-7027-4309-8e25-5f88f39f9d23 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:23.662440 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903a4689-bc18-49c1-85e8-bc86eee34625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:23.663312 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a4689-bc18-49c1-85e8-bc86eee34625 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:23.663598 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a4689-bc18-49c1-85e8-bc86eee34625 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:23.663905 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903a4689-bc18-49c1-85e8-bc86eee34625 None admin] PATH_INFO: `/v3/projects/aa8007020ec543c1a6405283ddd21067` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:23.665102 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903a4689-bc18-49c1-85e8-bc86eee34625 None admin] RBAC: Authorizing `identity:get_project(project_id=aa8007020ec543c1a6405283ddd21067)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:23.667074 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903a4689-bc18-49c1-85e8-bc86eee34625 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:23.669162 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 148/317] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:54:23 2022] GET /identity/v3/projects/aa8007020ec543c1a6405283ddd21067 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:23.674619 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cba74d-841f-4cf0-9d30-0711c3334ee4 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:23.785446 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2cba74d-841f-4cf0-9d30-0711c3334ee4 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:23.820558 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a497aa-ef4a-44a0-947c-d0aa2e9980b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:23.821220 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a497aa-ef4a-44a0-947c-d0aa2e9980b7 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:23.821507 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a497aa-ef4a-44a0-947c-d0aa2e9980b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:23.821781 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a497aa-ef4a-44a0-947c-d0aa2e9980b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:23.822635 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a497aa-ef4a-44a0-947c-d0aa2e9980b7 None admin] RBAC: Adding query filter params (role.id=e9f69ea47fce47fab94435de5a29de5f, scope.project.id=aa8007020ec543c1a6405283ddd21067, user.id=753f662c626846f5bb6592f912deb949) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:23.822949 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a497aa-ef4a-44a0-947c-d0aa2e9980b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:23.824419 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a497aa-ef4a-44a0-947c-d0aa2e9980b7 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:23.830726 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 170/318] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:54:23 2022] GET /identity/v3/role_assignments?user.id=753f662c626846f5bb6592f912deb949&scope.project.id=aa8007020ec543c1a6405283ddd21067&role.id=e9f69ea47fce47fab94435de5a29de5f => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:24.528189 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396bdfb7-ea55-4a21-ab00-f815c07b83f0 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:24.528701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396bdfb7-ea55-4a21-ab00-f815c07b83f0 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:24.528774 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396bdfb7-ea55-4a21-ab00-f815c07b83f0 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:24.530148 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 149/319] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:24.535352 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b6ff10-1de8-445c-b441-0682e959af27 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:24.535652 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b6ff10-1de8-445c-b441-0682e959af27 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:24.535964 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b6ff10-1de8-445c-b441-0682e959af27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:24.553785 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-57b6ff10-1de8-445c-b441-0682e959af27 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:24.554973 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-57b6ff10-1de8-445c-b441-0682e959af27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:24.575942 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969184ea-e270-4d14-9449-f2336cffb97d None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:24.576238 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969184ea-e270-4d14-9449-f2336cffb97d None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:24.576522 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969184ea-e270-4d14-9449-f2336cffb97d None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:24.577909 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 150/320] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 16:54:24.582630 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2a7085-9047-4b2a-843d-beb257337e42 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:24.582917 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2a7085-9047-4b2a-843d-beb257337e42 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:24.583197 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2a7085-9047-4b2a-843d-beb257337e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:24.588672 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:24.600209 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-2e2a7085-9047-4b2a-843d-beb257337e42 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:24.601225 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-2e2a7085-9047-4b2a-843d-beb257337e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:24.612865 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:24.626281 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 171/321] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:24.649172 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b6ff10-1de8-445c-b441-0682e959af27 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:24.672006 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 151/322] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 16:54:24.693996 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2a7085-9047-4b2a-843d-beb257337e42 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:24.763410 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b6ff10-1de8-445c-b441-0682e959af27 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:24.801304 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c77a820-1a2c-4cf9-aa6b-949ab0e552b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:24.802026 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77a820-1a2c-4cf9-aa6b-949ab0e552b6 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:24.802312 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77a820-1a2c-4cf9-aa6b-949ab0e552b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:24.802588 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c77a820-1a2c-4cf9-aa6b-949ab0e552b6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:24.807996 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2a7085-9047-4b2a-843d-beb257337e42 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:24.813191 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c77a820-1a2c-4cf9-aa6b-949ab0e552b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:24.820989 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-6c77a820-1a2c-4cf9-aa6b-949ab0e552b6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 16:54:24.820989 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:24.820989 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:24.820989 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:24.820989 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:24.820989 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:24.820989 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:24.820989 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:24.820989 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:24.820989 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:24.820989 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:24.820989 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:24.820989 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:24.820989 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:24.820989 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:24.820989 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:24.821842 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:24.821842 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:24.821842 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:24.821842 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:24.821842 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:24.821842 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:24.821842 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:24.821842 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:24.821842 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:24.821842 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:24.821842 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:24.821842 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:24.821842 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:24.821842 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:24.821842 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:24.821842 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:24.824389 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:24.824389 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:24.824389 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:24.824389 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:24.824389 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:24.824389 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:24.824389 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:24.824389 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:24.824389 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:24.824389 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:24.824389 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 16:54:24.824389 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:24.824389 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 172/323] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:54:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:54:24.828556 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c77a820-1a2c-4cf9-aa6b-949ab0e552b6 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:24.843608 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:24.848572 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d82c1-d859-4b4a-8ef6-5e4b22c33f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:24.849382 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d82c1-d859-4b4a-8ef6-5e4b22c33f0a None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:24.849669 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d82c1-d859-4b4a-8ef6-5e4b22c33f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:24.849944 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d82c1-d859-4b4a-8ef6-5e4b22c33f0a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:24.851100 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d82c1-d859-4b4a-8ef6-5e4b22c33f0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:24.852601 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d82c1-d859-4b4a-8ef6-5e4b22c33f0a None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:24.854838 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 152/324] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:54:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:24.860170 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d82c1-d859-4b4a-8ef6-5e4b22c33f0a None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:24.883795 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:24.939378 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c77a820-1a2c-4cf9-aa6b-949ab0e552b6 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:24.965942 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d82c1-d859-4b4a-8ef6-5e4b22c33f0a None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:24.977439 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb8fca8-aa5a-49db-9ef7-e930219aa134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:24.978115 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb8fca8-aa5a-49db-9ef7-e930219aa134 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:24.978401 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb8fca8-aa5a-49db-9ef7-e930219aa134 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:24.978710 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb8fca8-aa5a-49db-9ef7-e930219aa134 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:24.979466 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb8fca8-aa5a-49db-9ef7-e930219aa134 None admin] RBAC: Adding query filter params (name=member) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:24.979811 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb8fca8-aa5a-49db-9ef7-e930219aa134 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:24.981201 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb8fca8-aa5a-49db-9ef7-e930219aa134 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:24.990122 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 173/325] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 6 16:54:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:24.995499 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb8fca8-aa5a-49db-9ef7-e930219aa134 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:25.007158 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0d0991-3eba-4799-a67e-2eaa2922d034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:25.007798 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d0991-3eba-4799-a67e-2eaa2922d034 None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:25.008083 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d0991-3eba-4799-a67e-2eaa2922d034 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:25.008358 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0d0991-3eba-4799-a67e-2eaa2922d034 None admin] PATH_INFO: `/v3/groups` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:25.009051 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0d0991-3eba-4799-a67e-2eaa2922d034 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:25.011025 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0d0991-3eba-4799-a67e-2eaa2922d034 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:25.096705 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb8fca8-aa5a-49db-9ef7-e930219aa134 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:25.107039 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:25.114873 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 153/326] 10.222.0.40 () {60 vars in 1134 bytes} [Wed Jul 6 16:54:24 2022] POST /identity/v3/groups => generated 226 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 16:54:25.116369 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e4b26-0d13-4983-a317-5241a9bd548c None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:25.116661 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e4b26-0d13-4983-a317-5241a9bd548c None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:25.121997 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e4b26-0d13-4983-a317-5241a9bd548c None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:25.121997 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 154/327] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:25.125030 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d629dce-0ba4-4176-97a7-dd26d60007da None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:25.125334 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d629dce-0ba4-4176-97a7-dd26d60007da None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:25.125620 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d629dce-0ba4-4176-97a7-dd26d60007da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:25.131904 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d95c9fb-640a-4d8c-996a-6294978f8a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:25.132815 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95c9fb-640a-4d8c-996a-6294978f8a52 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:25.133097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95c9fb-640a-4d8c-996a-6294978f8a52 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:25.134260 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d95c9fb-640a-4d8c-996a-6294978f8a52 None admin] PATH_INFO: `/v3/users/8dfcf3bc25ef4136aba694857496a897` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:25.134663 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d95c9fb-640a-4d8c-996a-6294978f8a52 None admin] RBAC: Authorizing `identity:get_user(user_id=8dfcf3bc25ef4136aba694857496a897)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:25.136789 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d95c9fb-640a-4d8c-996a-6294978f8a52 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:25.138714 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 174/328] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:54:24 2022] GET /identity/v3/users/8dfcf3bc25ef4136aba694857496a897 => generated 293 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:25.144598 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d95c9fb-640a-4d8c-996a-6294978f8a52 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:25.146389 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-2d629dce-0ba4-4176-97a7-dd26d60007da None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:25.147619 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-2d629dce-0ba4-4176-97a7-dd26d60007da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:25.222150 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:25.236167 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 155/329] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 16:54:25.257712 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d629dce-0ba4-4176-97a7-dd26d60007da None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:25.261402 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d95c9fb-640a-4d8c-996a-6294978f8a52 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:25.300954 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b401a2-6807-4616-8bd0-8aec3542a949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:25.301774 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b401a2-6807-4616-8bd0-8aec3542a949 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:25.302138 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b401a2-6807-4616-8bd0-8aec3542a949 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:25.302441 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b401a2-6807-4616-8bd0-8aec3542a949 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:25.303474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b401a2-6807-4616-8bd0-8aec3542a949 None admin] RBAC: Adding query filter params (role.id=7fd63d2d99254c00bb3501173643ee30, user.id=8dfcf3bc25ef4136aba694857496a897, scope.system=all) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:25.303888 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b401a2-6807-4616-8bd0-8aec3542a949 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:25.305501 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b401a2-6807-4616-8bd0-8aec3542a949 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:25.319613 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 175/330] 10.222.0.40 () {56 vars in 1308 bytes} [Wed Jul 6 16:54:25 2022] GET /identity/v3/role_assignments?user.id=8dfcf3bc25ef4136aba694857496a897&scope.system=all&role.id=7fd63d2d99254c00bb3501173643ee30 => generated 510 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:25.383261 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d629dce-0ba4-4176-97a7-dd26d60007da None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:25.424161 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d67311-d480-4961-8dda-3f11ce7f4ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:25.425002 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d67311-d480-4961-8dda-3f11ce7f4ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:25.425294 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d67311-d480-4961-8dda-3f11ce7f4ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:25.425575 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d67311-d480-4961-8dda-3f11ce7f4ad0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:25.438504 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d67311-d480-4961-8dda-3f11ce7f4ad0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:25.446899 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-d5d67311-d480-4961-8dda-3f11ce7f4ad0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 16:54:25.446899 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:25.446899 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:25.446899 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:25.446899 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:25.446899 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:25.446899 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:25.446899 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:25.446899 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:25.446899 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:25.446899 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:25.446899 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:25.446899 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:25.446899 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:25.446899 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:25.446899 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:25.448506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:25.448506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:25.448506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:25.448506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:25.448506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:25.448506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:25.448506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:25.448506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:25.448506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:25.448506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:25.448506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:25.448506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:25.448506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:25.448506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:25.448506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:25.448506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:25.449290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:25.449290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:25.449290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:25.449290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:25.449290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:25.449290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:25.449290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:25.449290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:25.449290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:25.449290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:25.449290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 16:54:25.449290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:25.449290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 156/331] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:54:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:54:25.454989 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b401a2-6807-4616-8bd0-8aec3542a949 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:25.497535 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:25.575013 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b401a2-6807-4616-8bd0-8aec3542a949 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:25.614806 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d0570-6548-4887-bc78-10945972de8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:25.615486 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d0570-6548-4887-bc78-10945972de8e None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:25.615792 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d0570-6548-4887-bc78-10945972de8e None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:25.616875 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822d0570-6548-4887-bc78-10945972de8e None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:25.616875 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822d0570-6548-4887-bc78-10945972de8e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:25.617122 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822d0570-6548-4887-bc78-10945972de8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:25.618474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822d0570-6548-4887-bc78-10945972de8e None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:25.628381 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 176/332] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 6 16:54:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:25.634007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d67311-d480-4961-8dda-3f11ce7f4ad0 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:25.691644 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:25.755115 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d67311-d480-4961-8dda-3f11ce7f4ad0 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:25.793755 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c55ef-160c-4d8c-8226-a8d3acc43087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:25.794656 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c55ef-160c-4d8c-8226-a8d3acc43087 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:25.794942 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c55ef-160c-4d8c-8226-a8d3acc43087 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:25.795221 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c55ef-160c-4d8c-8226-a8d3acc43087 None admin] PATH_INFO: `/v3/users/2f07f64cea16495d9eea180957631259` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:25.796478 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c55ef-160c-4d8c-8226-a8d3acc43087 None admin] RBAC: Authorizing `identity:get_user(user_id=2f07f64cea16495d9eea180957631259)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:25.797957 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c55ef-160c-4d8c-8226-a8d3acc43087 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:25.799952 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 157/333] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:54:25 2022] GET /identity/v3/users/2f07f64cea16495d9eea180957631259 => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:25.805524 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d0570-6548-4887-bc78-10945972de8e None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:25.920590 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822d0570-6548-4887-bc78-10945972de8e None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:25.923293 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:25.961173 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d585098-aad7-44db-b69f-d2bf42199013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:25.962066 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d585098-aad7-44db-b69f-d2bf42199013 None admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:25.962356 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d585098-aad7-44db-b69f-d2bf42199013 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:25.962639 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d585098-aad7-44db-b69f-d2bf42199013 None admin] PATH_INFO: `/v3/system/users/2f07f64cea16495d9eea180957631259/roles/e9f69ea47fce47fab94435de5a29de5f` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:25.964957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d585098-aad7-44db-b69f-d2bf42199013 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2f07f64cea16495d9eea180957631259, role_id=e9f69ea47fce47fab94435de5a29de5f)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:25.966414 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d585098-aad7-44db-b69f-d2bf42199013 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:26.059223 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 177/334] 10.222.0.40 () {58 vars in 1334 bytes} [Wed Jul 6 16:54:25 2022] PUT /identity/v3/system/users/2f07f64cea16495d9eea180957631259/roles/e9f69ea47fce47fab94435de5a29de5f => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 16:54:27.811919 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a18e1c-35e2-4663-bae0-b784a5d2d222 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:27.812459 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a18e1c-35e2-4663-bae0-b784a5d2d222 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:27.812858 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a18e1c-35e2-4663-bae0-b784a5d2d222 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:27.814489 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 158/335] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:27.820096 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee0cec-25d6-4e7b-935d-7033a429298f None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:27.820953 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee0cec-25d6-4e7b-935d-7033a429298f None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:27.821458 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee0cec-25d6-4e7b-935d-7033a429298f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:27.844785 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-4fee0cec-25d6-4e7b-935d-7033a429298f None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:27.846431 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-4fee0cec-25d6-4e7b-935d-7033a429298f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:27.929890 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 178/336] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:27.956067 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd31d3-4e4e-434c-b4ef-7f5f58fdebd9 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:27.956708 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd31d3-4e4e-434c-b4ef-7f5f58fdebd9 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:27.957199 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dd31d3-4e4e-434c-b4ef-7f5f58fdebd9 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:27.958804 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 159/337] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:27.964872 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf3603-9824-423d-bd05-793f24286693 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:27.965416 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf3603-9824-423d-bd05-793f24286693 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:27.965902 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cf3603-9824-423d-bd05-793f24286693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:27.969667 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dd31d3-4e4e-434c-b4ef-7f5f58fdebd9 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:27.997180 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-e3cf3603-9824-423d-bd05-793f24286693 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:27.998324 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-e3cf3603-9824-423d-bd05-793f24286693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:28.098347 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 179/338] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:28.108063 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dd31d3-4e4e-434c-b4ef-7f5f58fdebd9 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:28.132602 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:28.139114 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf3603-9824-423d-bd05-793f24286693 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:28.158926 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abef48ab-4bfc-442f-bc95-901b89148079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:28.159774 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef48ab-4bfc-442f-bc95-901b89148079 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:28.160109 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef48ab-4bfc-442f-bc95-901b89148079 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:28.160442 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abef48ab-4bfc-442f-bc95-901b89148079 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:28.174382 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:28.177725 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abef48ab-4bfc-442f-bc95-901b89148079 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:28.185841 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-abef48ab-4bfc-442f-bc95-901b89148079 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 16:54:28.185841 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:28.185841 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:28.185841 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:28.185841 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:28.185841 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:28.185841 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:28.185841 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:28.185841 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:28.185841 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:28.185841 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:28.185841 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:28.185841 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:28.185841 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:28.185841 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:28.185841 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:28.186730 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:28.186730 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:28.186730 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:28.186730 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:28.186730 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:28.186730 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:28.186730 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:28.186730 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:28.186730 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:28.186730 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:28.186730 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:28.186730 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:28.186730 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:28.186730 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:28.186730 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:28.186730 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:28.187506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:28.187506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:28.187506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:28.187506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:28.187506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:28.187506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:28.187506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:28.187506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:28.187506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:28.187506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:28.187506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 16:54:28.187506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:28.192037 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 160/339] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:54:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:28.208771 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abef48ab-4bfc-442f-bc95-901b89148079 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:28.329659 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cf3603-9824-423d-bd05-793f24286693 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:28.349249 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abef48ab-4bfc-442f-bc95-901b89148079 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:28.386198 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b86812-2e58-44bd-8586-8308f564580c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:28.391766 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b86812-2e58-44bd-8586-8308f564580c None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:28.392515 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b86812-2e58-44bd-8586-8308f564580c None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:28.392968 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b86812-2e58-44bd-8586-8308f564580c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:28.403143 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ea057-762f-43fc-83d1-0f16143747af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:28.407749 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ea057-762f-43fc-83d1-0f16143747af None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:28.407749 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ea057-762f-43fc-83d1-0f16143747af None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:28.407749 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ea057-762f-43fc-83d1-0f16143747af None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:28.407749 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ea057-762f-43fc-83d1-0f16143747af None admin] RBAC: Adding query filter params (name=member) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:28.407749 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ea057-762f-43fc-83d1-0f16143747af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:28.407749 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ea057-762f-43fc-83d1-0f16143747af None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:28.412722 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b86812-2e58-44bd-8586-8308f564580c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:28.420535 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 161/340] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 6 16:54:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:28.425557 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ea057-762f-43fc-83d1-0f16143747af None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:28.425682 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-18b86812-2e58-44bd-8586-8308f564580c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 16:54:28.425682 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:28.425682 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:28.425682 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:28.425682 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:28.425682 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:28.425682 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:28.425682 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:28.425682 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:28.425682 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:28.425682 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:28.425682 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:28.425682 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:28.425682 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:28.425682 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:28.425682 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:28.426491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:28.426491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:28.426491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:28.426491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:28.426491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:28.426491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:28.426491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:28.426491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:28.426491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:28.426491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:28.426491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:28.426491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:28.426491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:28.426491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:28.426491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:28.426491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:28.427270 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:28.427270 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:28.427270 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:28.427270 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:28.427270 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:28.427270 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:28.427270 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:28.427270 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:28.427270 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:28.427270 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:28.427270 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 16:54:28.427270 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:28.427270 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 180/341] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:54:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:28.433000 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b86812-2e58-44bd-8586-8308f564580c None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:28.485855 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:28.555686 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ea057-762f-43fc-83d1-0f16143747af None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:28.592406 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8895e259-d9f4-4f31-8f8b-54dd3e186fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:28.593298 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895e259-d9f4-4f31-8f8b-54dd3e186fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:28.593734 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895e259-d9f4-4f31-8f8b-54dd3e186fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:28.594177 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8895e259-d9f4-4f31-8f8b-54dd3e186fb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:28.599519 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:28.637631 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b86812-2e58-44bd-8586-8308f564580c None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:28.668783 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8895e259-d9f4-4f31-8f8b-54dd3e186fb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:28.670495 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8895e259-d9f4-4f31-8f8b-54dd3e186fb3 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:28.699974 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea48a4d4-204c-41d6-a727-0b9e267985ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:28.700653 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48a4d4-204c-41d6-a727-0b9e267985ec None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:28.700935 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48a4d4-204c-41d6-a727-0b9e267985ec None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:28.701206 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48a4d4-204c-41d6-a727-0b9e267985ec None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:28.702064 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea48a4d4-204c-41d6-a727-0b9e267985ec None admin] RBAC: Adding query filter params (name=member) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:28.702374 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea48a4d4-204c-41d6-a727-0b9e267985ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:28.703895 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea48a4d4-204c-41d6-a727-0b9e267985ec None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:28.714559 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:28.736531 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 181/342] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 6 16:54:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:28.738117 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e4154-e296-4239-ae44-5b3547b4c106 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:28.743883 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e4154-e296-4239-ae44-5b3547b4c106 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:28.743883 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460e4154-e296-4239-ae44-5b3547b4c106 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:28.745725 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 182/343] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:28.746649 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e4154-e296-4239-ae44-5b3547b4c106 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:28.787561 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 162/344] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:54:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 06 16:54:28.789007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acda253-e36b-4762-909d-d890429f2920 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:28.789305 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acda253-e36b-4762-909d-d890429f2920 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:28.789588 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acda253-e36b-4762-909d-d890429f2920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:28.816368 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-4acda253-e36b-4762-909d-d890429f2920 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:28.823243 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4acda253-e36b-4762-909d-d890429f2920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:28.946804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460e4154-e296-4239-ae44-5b3547b4c106 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:28.951547 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 163/345] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:28.952447 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acda253-e36b-4762-909d-d890429f2920 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:29.018052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247a83e1-5f51-4b5e-abb6-e6c4249cb856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:29.018772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a83e1-5f51-4b5e-abb6-e6c4249cb856 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:29.019065 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a83e1-5f51-4b5e-abb6-e6c4249cb856 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:29.019346 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247a83e1-5f51-4b5e-abb6-e6c4249cb856 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:29.054534 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:29.074170 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:29.121580 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acda253-e36b-4762-909d-d890429f2920 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:29.133539 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247a83e1-5f51-4b5e-abb6-e6c4249cb856 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:29.135655 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247a83e1-5f51-4b5e-abb6-e6c4249cb856 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:29.162416 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4488f033-043c-4598-9d9f-f73ee208cd14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:29.163295 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4488f033-043c-4598-9d9f-f73ee208cd14 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:29.163589 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4488f033-043c-4598-9d9f-f73ee208cd14 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:29.163896 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4488f033-043c-4598-9d9f-f73ee208cd14 None admin] PATH_INFO: `/v3/projects/aa8007020ec543c1a6405283ddd21067` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:29.167587 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4488f033-043c-4598-9d9f-f73ee208cd14 None admin] RBAC: Authorizing `identity:get_project(project_id=aa8007020ec543c1a6405283ddd21067)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:29.169220 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4488f033-043c-4598-9d9f-f73ee208cd14 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:29.170997 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 164/346] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:54:28 2022] GET /identity/v3/projects/aa8007020ec543c1a6405283ddd21067 => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:29.171799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4488f033-043c-4598-9d9f-f73ee208cd14 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:29.250585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 183/347] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:54:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 504 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 16:54:29.252404 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dfb8f9-24c2-47b8-bcf6-04a96d8b576c None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:29.252840 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dfb8f9-24c2-47b8-bcf6-04a96d8b576c None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:29.253284 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dfb8f9-24c2-47b8-bcf6-04a96d8b576c None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:29.258925 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 184/348] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:29.260541 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dfb8f9-24c2-47b8-bcf6-04a96d8b576c None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:29.283027 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4488f033-043c-4598-9d9f-f73ee208cd14 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:29.316111 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:29.320381 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd02242-6276-44e7-94c1-312cc13092b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:29.321346 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd02242-6276-44e7-94c1-312cc13092b7 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:29.321736 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd02242-6276-44e7-94c1-312cc13092b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:29.322120 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd02242-6276-44e7-94c1-312cc13092b7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:29.330541 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:29.334337 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd02242-6276-44e7-94c1-312cc13092b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:29.343834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-bbd02242-6276-44e7-94c1-312cc13092b7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 16:54:29.343834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:29.343834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:29.343834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:29.343834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:29.343834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:29.343834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:29.343834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:29.343834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:29.343834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:29.343834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:29.343834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:29.343834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:29.343834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:29.343834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:29.343834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:29.344778 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:29.344778 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:29.344778 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:29.344778 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:29.344778 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:29.344778 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:29.344778 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:29.344778 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:29.344778 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:29.344778 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:29.344778 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:29.344778 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:29.344778 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:29.344778 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:29.344778 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:29.344778 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:29.345596 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:29.345596 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:29.345596 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:29.345596 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:29.345596 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:29.345596 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:29.345596 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:29.345596 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:29.345596 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:29.345596 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:29.345596 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 16:54:29.345596 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:29.348444 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 165/349] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 6 16:54:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:29.349548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd02242-6276-44e7-94c1-312cc13092b7 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:29.379105 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dfb8f9-24c2-47b8-bcf6-04a96d8b576c None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:29.425361 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d24347-334c-4518-93e6-18bd638428e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:29.426567 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d24347-334c-4518-93e6-18bd638428e0 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:29.426996 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d24347-334c-4518-93e6-18bd638428e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:29.427418 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d24347-334c-4518-93e6-18bd638428e0 None admin] PATH_INFO: `/v3/groups/1690f24ae1274d30b06a3c3e5c02cf6f` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:29.438368 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d24347-334c-4518-93e6-18bd638428e0 None admin] RBAC: Authorizing `identity:get_group(group_id=1690f24ae1274d30b06a3c3e5c02cf6f)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:29.438602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d24347-334c-4518-93e6-18bd638428e0 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:29.440726 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 185/350] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 6 16:54:29 2022] GET /identity/v3/groups/1690f24ae1274d30b06a3c3e5c02cf6f => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:29.444906 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d06e4b-cb80-4286-8b91-3f4610b7abdf None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:29.444906 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d06e4b-cb80-4286-8b91-3f4610b7abdf None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:29.445122 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d06e4b-cb80-4286-8b91-3f4610b7abdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:29.467419 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-31d06e4b-cb80-4286-8b91-3f4610b7abdf None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:29.468609 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-31d06e4b-cb80-4286-8b91-3f4610b7abdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:29.479993 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd02242-6276-44e7-94c1-312cc13092b7 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:29.517846 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b145e20-45c2-4781-87d2-068100b975f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:29.518636 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b145e20-45c2-4781-87d2-068100b975f8 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:29.519124 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b145e20-45c2-4781-87d2-068100b975f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:29.519593 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b145e20-45c2-4781-87d2-068100b975f8 None admin] PATH_INFO: `/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:29.521005 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b145e20-45c2-4781-87d2-068100b975f8 None admin] RBAC: Authorizing `identity:get_project(project_id=49b1ed5ea8fc417f8247e689e804e6f1)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:29.522931 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b145e20-45c2-4781-87d2-068100b975f8 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:29.525742 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 166/351] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:54:29 2022] GET /identity/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:29.526682 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b145e20-45c2-4781-87d2-068100b975f8 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:29.555540 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 186/352] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:29.557679 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b566c-4be6-4c08-9c1e-d282f2fc329a None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:29.557969 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b566c-4be6-4c08-9c1e-d282f2fc329a None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:29.558268 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b566c-4be6-4c08-9c1e-d282f2fc329a None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:29.563889 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 187/353] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:29.564790 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b566c-4be6-4c08-9c1e-d282f2fc329a None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:29.608287 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:29.620070 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:29.652715 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b145e20-45c2-4781-87d2-068100b975f8 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:29.692513 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0591b513-f0a8-4f00-b7a7-26b82ef1c704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:29.693196 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591b513-f0a8-4f00-b7a7-26b82ef1c704 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:29.693475 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591b513-f0a8-4f00-b7a7-26b82ef1c704 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:29.693756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591b513-f0a8-4f00-b7a7-26b82ef1c704 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:29.694041 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b566c-4be6-4c08-9c1e-d282f2fc329a None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:29.694517 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0591b513-f0a8-4f00-b7a7-26b82ef1c704 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:29.694857 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0591b513-f0a8-4f00-b7a7-26b82ef1c704 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:29.696570 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0591b513-f0a8-4f00-b7a7-26b82ef1c704 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:29.705974 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 167/354] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:54:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:29.706811 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0591b513-f0a8-4f00-b7a7-26b82ef1c704 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:29.747114 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8edad39-a866-4d5e-a906-a4502b82c132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:29.747968 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8edad39-a866-4d5e-a906-a4502b82c132 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:29.748191 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8edad39-a866-4d5e-a906-a4502b82c132 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:29.748463 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8edad39-a866-4d5e-a906-a4502b82c132 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:29.749300 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8edad39-a866-4d5e-a906-a4502b82c132 None admin] RBAC: Adding query filter params (group.id=1690f24ae1274d30b06a3c3e5c02cf6f, role.id=7fd63d2d99254c00bb3501173643ee30, scope.project.id=aa8007020ec543c1a6405283ddd21067) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:29.749618 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8edad39-a866-4d5e-a906-a4502b82c132 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:29.751760 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8edad39-a866-4d5e-a906-a4502b82c132 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:29.757473 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 188/355] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Jul 6 16:54:29 2022] GET /identity/v3/role_assignments?group.id=1690f24ae1274d30b06a3c3e5c02cf6f&scope.project.id=aa8007020ec543c1a6405283ddd21067&role.id=7fd63d2d99254c00bb3501173643ee30 => generated 260 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:29.758938 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14967274-2813-434c-9c00-b4bb99511003 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:29.759054 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14967274-2813-434c-9c00-b4bb99511003 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:29.759314 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14967274-2813-434c-9c00-b4bb99511003 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:29.778256 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-14967274-2813-434c-9c00-b4bb99511003 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:29.779345 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-14967274-2813-434c-9c00-b4bb99511003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:29.849007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0591b513-f0a8-4f00-b7a7-26b82ef1c704 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:29.872298 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 189/356] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:29.876284 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14967274-2813-434c-9c00-b4bb99511003 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:29.882253 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:29.891433 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91bf38a-59f8-44af-9ea0-70637987d23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:29.892402 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91bf38a-59f8-44af-9ea0-70637987d23c None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:29.892686 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91bf38a-59f8-44af-9ea0-70637987d23c None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:29.892960 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91bf38a-59f8-44af-9ea0-70637987d23c None admin] PATH_INFO: `/v3/groups/1690f24ae1274d30b06a3c3e5c02cf6f` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:29.894521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91bf38a-59f8-44af-9ea0-70637987d23c None admin] RBAC: Authorizing `identity:get_group(group_id=1690f24ae1274d30b06a3c3e5c02cf6f)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:29.895844 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91bf38a-59f8-44af-9ea0-70637987d23c None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:29.898146 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 168/357] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 6 16:54:29 2022] GET /identity/v3/groups/1690f24ae1274d30b06a3c3e5c02cf6f => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:29.899484 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb49e77-1003-4c64-ba87-45a22b4f6622 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:29.899789 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb49e77-1003-4c64-ba87-45a22b4f6622 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:29.900065 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb49e77-1003-4c64-ba87-45a22b4f6622 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:29.902535 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 169/358] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:29.903285 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb49e77-1003-4c64-ba87-45a22b4f6622 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:29.926341 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:30.008847 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14967274-2813-434c-9c00-b4bb99511003 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:30.034237 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb49e77-1003-4c64-ba87-45a22b4f6622 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:30.049678 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aff801-7740-4868-b387-5de9e517c2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:30.050446 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aff801-7740-4868-b387-5de9e517c2c8 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:30.050730 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aff801-7740-4868-b387-5de9e517c2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:30.051014 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aff801-7740-4868-b387-5de9e517c2c8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:30.068428 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aff801-7740-4868-b387-5de9e517c2c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:30.073467 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fdfd18-17e8-422b-bd3d-79fe3649f34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:30.074124 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fdfd18-17e8-422b-bd3d-79fe3649f34d None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:30.074405 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fdfd18-17e8-422b-bd3d-79fe3649f34d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:30.074681 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fdfd18-17e8-422b-bd3d-79fe3649f34d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:30.078180 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-82aff801-7740-4868-b387-5de9e517c2c8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 16:54:30.078180 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:30.078180 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:30.078180 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:30.078180 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:30.078180 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:30.078180 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:30.078180 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:30.078180 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:30.078180 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:30.078180 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:30.078180 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:30.078180 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:30.078180 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:30.078180 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:30.078180 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:30.079087 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:30.079087 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:30.079087 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:30.079087 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:30.079087 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:30.079087 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:30.079087 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:30.079087 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:30.079087 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:30.079087 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:30.079087 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:30.079087 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:30.079087 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:30.079087 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:30.079087 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:30.079087 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:30.081240 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:30.081240 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:30.081240 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:30.081240 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:30.081240 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:30.081240 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:30.081240 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:30.081240 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:30.081240 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:30.081240 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:30.081240 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 06 16:54:30.081240 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:30.081240 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 190/359] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:54:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:30.081240 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aff801-7740-4868-b387-5de9e517c2c8 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:30.155334 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:30.165502 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fdfd18-17e8-422b-bd3d-79fe3649f34d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:30.167212 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fdfd18-17e8-422b-bd3d-79fe3649f34d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:30.196322 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:30.208348 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aff801-7740-4868-b387-5de9e517c2c8 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:30.254976 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3092dd5-db52-4e0a-8b55-741f5f97ca18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:30.256412 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3092dd5-db52-4e0a-8b55-741f5f97ca18 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:30.256412 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3092dd5-db52-4e0a-8b55-741f5f97ca18 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:30.256601 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3092dd5-db52-4e0a-8b55-741f5f97ca18 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:30.273749 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3092dd5-db52-4e0a-8b55-741f5f97ca18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:30.283037 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-b3092dd5-db52-4e0a-8b55-741f5f97ca18 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 16:54:30.283037 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:30.283037 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:30.283037 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:30.283037 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:30.283037 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:30.283037 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:30.283037 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:30.283037 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:30.283037 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:30.283037 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:30.283037 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:30.283037 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:30.283037 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:30.283037 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:30.283037 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:30.284076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:30.284076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:30.284076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:30.284076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:30.284076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:30.284076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:30.284076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:30.284076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:30.284076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:30.284076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:30.284076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:30.284076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:30.284076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:30.284076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:30.284076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:30.284076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:30.284905 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:30.284905 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:30.284905 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:30.284905 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:30.284905 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:30.284905 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:30.284905 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:30.284905 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:30.284905 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:30.284905 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:30.284905 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 16:54:30.284905 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:30.284905 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 191/360] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 6 16:54:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:30.285582 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f9668-067f-4092-8c80-274796e7fc87 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:30.285913 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f9668-067f-4092-8c80-274796e7fc87 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:30.286337 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f9668-067f-4092-8c80-274796e7fc87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:30.304309 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-990f9668-067f-4092-8c80-274796e7fc87 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:30.304585 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 170/361] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:54:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 16:54:30.305482 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-990f9668-067f-4092-8c80-274796e7fc87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:30.305641 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fdfd18-17e8-422b-bd3d-79fe3649f34d None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:30.385818 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 192/362] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:30.386891 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f9668-067f-4092-8c80-274796e7fc87 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:30.430338 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:30.436264 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fdfd18-17e8-422b-bd3d-79fe3649f34d None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:30.479685 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04778b7-52a1-4fd0-9dae-7f2ea1063c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:30.481370 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04778b7-52a1-4fd0-9dae-7f2ea1063c64 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:30.481655 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04778b7-52a1-4fd0-9dae-7f2ea1063c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:30.481938 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04778b7-52a1-4fd0-9dae-7f2ea1063c64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:30.482783 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04778b7-52a1-4fd0-9dae-7f2ea1063c64 None admin] RBAC: Adding query filter params (group.id=1690f24ae1274d30b06a3c3e5c02cf6f, role.id=7fd63d2d99254c00bb3501173643ee30, scope.project.id=49b1ed5ea8fc417f8247e689e804e6f1) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:30.483100 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04778b7-52a1-4fd0-9dae-7f2ea1063c64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:30.484214 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:30.484626 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04778b7-52a1-4fd0-9dae-7f2ea1063c64 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:30.492764 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 171/363] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Jul 6 16:54:30 2022] GET /identity/v3/role_assignments?group.id=1690f24ae1274d30b06a3c3e5c02cf6f&scope.project.id=49b1ed5ea8fc417f8247e689e804e6f1&role.id=7fd63d2d99254c00bb3501173643ee30 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:30.501138 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04778b7-52a1-4fd0-9dae-7f2ea1063c64 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:30.505585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f9668-067f-4092-8c80-274796e7fc87 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:30.555523 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8058c1da-56ff-419c-95d3-6f5ebbd2420c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:30.557109 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058c1da-56ff-419c-95d3-6f5ebbd2420c None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:30.557603 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058c1da-56ff-419c-95d3-6f5ebbd2420c None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:30.558057 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8058c1da-56ff-419c-95d3-6f5ebbd2420c None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:30.559039 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8058c1da-56ff-419c-95d3-6f5ebbd2420c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:30.559458 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8058c1da-56ff-419c-95d3-6f5ebbd2420c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:30.561038 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8058c1da-56ff-419c-95d3-6f5ebbd2420c None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:30.570580 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 193/364] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 6 16:54:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:30.572835 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8058c1da-56ff-419c-95d3-6f5ebbd2420c None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:30.624541 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04778b7-52a1-4fd0-9dae-7f2ea1063c64 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:30.670461 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e464b0-3276-4b21-9be5-29d68c7a7d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:30.671264 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e464b0-3276-4b21-9be5-29d68c7a7d5c None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:30.671552 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e464b0-3276-4b21-9be5-29d68c7a7d5c None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:30.671858 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e464b0-3276-4b21-9be5-29d68c7a7d5c None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:30.672657 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e464b0-3276-4b21-9be5-29d68c7a7d5c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:30.672941 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e464b0-3276-4b21-9be5-29d68c7a7d5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:30.674338 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e464b0-3276-4b21-9be5-29d68c7a7d5c None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:30.684633 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 172/365] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:54:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:30.685576 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e464b0-3276-4b21-9be5-29d68c7a7d5c None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:30.699909 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8058c1da-56ff-419c-95d3-6f5ebbd2420c None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:30.713915 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:30.764402 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c170d95-2ba1-4160-8689-94a2c2cee0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:30.765288 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c170d95-2ba1-4160-8689-94a2c2cee0a6 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:30.765581 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c170d95-2ba1-4160-8689-94a2c2cee0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:30.765865 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c170d95-2ba1-4160-8689-94a2c2cee0a6 None admin] PATH_INFO: `/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:30.768593 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c170d95-2ba1-4160-8689-94a2c2cee0a6 None admin] RBAC: Authorizing `identity:get_project(project_id=49b1ed5ea8fc417f8247e689e804e6f1)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:30.769509 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c170d95-2ba1-4160-8689-94a2c2cee0a6 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:30.771690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 194/366] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:54:30 2022] GET /identity/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1 => generated 298 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:30.772390 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c170d95-2ba1-4160-8689-94a2c2cee0a6 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:30.792705 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:30.821478 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e464b0-3276-4b21-9be5-29d68c7a7d5c None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:30.854895 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d105d-3851-4222-b3ee-9c5f31b2db3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:30.855681 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d105d-3851-4222-b3ee-9c5f31b2db3f None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:30.856003 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d105d-3851-4222-b3ee-9c5f31b2db3f None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:30.856284 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051d105d-3851-4222-b3ee-9c5f31b2db3f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:30.866967 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051d105d-3851-4222-b3ee-9c5f31b2db3f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:30.876933 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-051d105d-3851-4222-b3ee-9c5f31b2db3f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:54:30.876933 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:30.876933 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:30.876933 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:30.876933 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:30.876933 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:30.876933 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:30.876933 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:30.876933 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:30.876933 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:30.876933 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:30.876933 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:30.876933 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:30.876933 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:30.876933 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:30.876933 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:30.877850 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:30.877850 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:30.877850 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:30.877850 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:30.877850 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:30.877850 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:30.877850 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:30.877850 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:30.877850 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:30.877850 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:30.877850 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:30.877850 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:30.877850 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:30.877850 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:30.877850 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:30.877850 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:30.878640 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:30.878640 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:30.878640 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:30.878640 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:30.878640 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:30.878640 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:30.878640 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:30.878640 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:30.878640 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:30.878640 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:30.878640 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:54:30.878640 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:30.878640 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 173/367] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 6 16:54:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:30.884084 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d105d-3851-4222-b3ee-9c5f31b2db3f None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:30.895932 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c170d95-2ba1-4160-8689-94a2c2cee0a6 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:30.936641 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35a46c5-6032-41d8-8860-558e9e76dc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:30.937530 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a46c5-6032-41d8-8860-558e9e76dc61 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:30.937817 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a46c5-6032-41d8-8860-558e9e76dc61 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:30.938097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35a46c5-6032-41d8-8860-558e9e76dc61 None admin] PATH_INFO: `/v3/users/2f07f64cea16495d9eea180957631259` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:30.940053 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35a46c5-6032-41d8-8860-558e9e76dc61 None admin] RBAC: Authorizing `identity:get_user(user_id=2f07f64cea16495d9eea180957631259)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:30.941516 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35a46c5-6032-41d8-8860-558e9e76dc61 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:30.943616 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 195/368] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:54:30 2022] GET /identity/v3/users/2f07f64cea16495d9eea180957631259 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:30.944418 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35a46c5-6032-41d8-8860-558e9e76dc61 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:30.996164 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:31.011011 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051d105d-3851-4222-b3ee-9c5f31b2db3f None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:31.042971 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:31.055310 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739b7ca1-3971-4db5-be7a-76ae5a9d93fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:31.055558 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b7ca1-3971-4db5-be7a-76ae5a9d93fd None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:31.055842 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b7ca1-3971-4db5-be7a-76ae5a9d93fd None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:31.056185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739b7ca1-3971-4db5-be7a-76ae5a9d93fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:31.075163 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35a46c5-6032-41d8-8860-558e9e76dc61 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:31.123230 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9e67f4-a4d4-4b21-a502-a4e9e6a4d30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:31.124979 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e67f4-a4d4-4b21-a502-a4e9e6a4d30d None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:31.125717 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e67f4-a4d4-4b21-a502-a4e9e6a4d30d None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:31.126146 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9e67f4-a4d4-4b21-a502-a4e9e6a4d30d None admin] PATH_INFO: `/v3/groups/1690f24ae1274d30b06a3c3e5c02cf6f` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:31.128054 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9e67f4-a4d4-4b21-a502-a4e9e6a4d30d None admin] RBAC: Authorizing `identity:get_group(group_id=1690f24ae1274d30b06a3c3e5c02cf6f)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:31.130134 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9e67f4-a4d4-4b21-a502-a4e9e6a4d30d None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:31.132941 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 196/369] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 6 16:54:30 2022] GET /identity/v3/groups/1690f24ae1274d30b06a3c3e5c02cf6f => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:31.133944 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9e67f4-a4d4-4b21-a502-a4e9e6a4d30d None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:31.145783 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739b7ca1-3971-4db5-be7a-76ae5a9d93fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:31.148187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739b7ca1-3971-4db5-be7a-76ae5a9d93fd None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:31.240344 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:31.260995 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 174/370] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:54:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 16:54:31.262663 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739b7ca1-3971-4db5-be7a-76ae5a9d93fd None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:31.287092 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9e67f4-a4d4-4b21-a502-a4e9e6a4d30d None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:31.324739 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438464a9-de2b-4ed2-9f6d-146fd8ab3938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:31.324739 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438464a9-de2b-4ed2-9f6d-146fd8ab3938 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:31.325149 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438464a9-de2b-4ed2-9f6d-146fd8ab3938 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:31.325375 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438464a9-de2b-4ed2-9f6d-146fd8ab3938 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:31.326081 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438464a9-de2b-4ed2-9f6d-146fd8ab3938 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:31.326390 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438464a9-de2b-4ed2-9f6d-146fd8ab3938 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:31.328527 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438464a9-de2b-4ed2-9f6d-146fd8ab3938 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:31.341316 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 197/371] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:54:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:31.341316 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438464a9-de2b-4ed2-9f6d-146fd8ab3938 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:31.359021 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:31.407260 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739b7ca1-3971-4db5-be7a-76ae5a9d93fd None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:31.453024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd561f5-5cf6-4832-baf4-e865c6c0644c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:31.454178 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd561f5-5cf6-4832-baf4-e865c6c0644c None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:31.454987 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd561f5-5cf6-4832-baf4-e865c6c0644c None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:31.455489 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd561f5-5cf6-4832-baf4-e865c6c0644c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:31.456827 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd561f5-5cf6-4832-baf4-e865c6c0644c None admin] RBAC: Adding query filter params (role.id=e9f69ea47fce47fab94435de5a29de5f, user.id=2f07f64cea16495d9eea180957631259, scope.system=all) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:31.457348 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd561f5-5cf6-4832-baf4-e865c6c0644c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:31.459780 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd561f5-5cf6-4832-baf4-e865c6c0644c None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:31.461213 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438464a9-de2b-4ed2-9f6d-146fd8ab3938 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:31.475941 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 175/372] 10.222.0.40 () {56 vars in 1308 bytes} [Wed Jul 6 16:54:31 2022] GET /identity/v3/role_assignments?user.id=2f07f64cea16495d9eea180957631259&scope.system=all&role.id=e9f69ea47fce47fab94435de5a29de5f => generated 510 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:31.477685 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd561f5-5cf6-4832-baf4-e865c6c0644c None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:31.503583 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d66ee2c-8af2-4090-9ad5-fb23f2248acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:31.504313 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d66ee2c-8af2-4090-9ad5-fb23f2248acf None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:31.504604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d66ee2c-8af2-4090-9ad5-fb23f2248acf None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:31.504883 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d66ee2c-8af2-4090-9ad5-fb23f2248acf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:31.505749 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d66ee2c-8af2-4090-9ad5-fb23f2248acf None admin] RBAC: Adding query filter params (group.id=1690f24ae1274d30b06a3c3e5c02cf6f, role.id=c31c5ee7ed8646e48fa81e85d10fcaf2, scope.project.id=49b1ed5ea8fc417f8247e689e804e6f1) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:31.506069 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d66ee2c-8af2-4090-9ad5-fb23f2248acf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:31.507795 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d66ee2c-8af2-4090-9ad5-fb23f2248acf None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:31.519769 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 198/373] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Jul 6 16:54:31 2022] GET /identity/v3/role_assignments?group.id=1690f24ae1274d30b06a3c3e5c02cf6f&scope.project.id=49b1ed5ea8fc417f8247e689e804e6f1&role.id=c31c5ee7ed8646e48fa81e85d10fcaf2 => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:31.519769 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d66ee2c-8af2-4090-9ad5-fb23f2248acf None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:31.559320 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:31.677885 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d66ee2c-8af2-4090-9ad5-fb23f2248acf None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:31.678638 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd561f5-5cf6-4832-baf4-e865c6c0644c None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:31.680926 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:31.716300 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e186d656-3427-4209-9d16-c7b825fe9eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:31.716968 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e186d656-3427-4209-9d16-c7b825fe9eca None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:31.717257 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e186d656-3427-4209-9d16-c7b825fe9eca None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:31.717537 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e186d656-3427-4209-9d16-c7b825fe9eca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:31.739282 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3ac5eb-3bd1-411f-89ae-9629719aa837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:31.739282 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ac5eb-3bd1-411f-89ae-9629719aa837 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:31.739282 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ac5eb-3bd1-411f-89ae-9629719aa837 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:31.739282 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3ac5eb-3bd1-411f-89ae-9629719aa837 None admin] PATH_INFO: `/v3/projects/aa8007020ec543c1a6405283ddd21067` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:31.739906 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3ac5eb-3bd1-411f-89ae-9629719aa837 None admin] RBAC: Authorizing `identity:get_project(project_id=aa8007020ec543c1a6405283ddd21067)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:31.740773 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3ac5eb-3bd1-411f-89ae-9629719aa837 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:31.742845 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 176/374] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:54:31 2022] GET /identity/v3/projects/aa8007020ec543c1a6405283ddd21067 => generated 302 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:31.749763 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3ac5eb-3bd1-411f-89ae-9629719aa837 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:31.793886 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e186d656-3427-4209-9d16-c7b825fe9eca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:31.795891 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e186d656-3427-4209-9d16-c7b825fe9eca None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:31.874396 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3ac5eb-3bd1-411f-89ae-9629719aa837 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:31.902826 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 199/375] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:54:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 16:54:31.905397 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:31.908056 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e186d656-3427-4209-9d16-c7b825fe9eca None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:31.911138 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee020d-5d2f-4067-9a29-4557e54d8361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:31.912293 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee020d-5d2f-4067-9a29-4557e54d8361 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:31.912806 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee020d-5d2f-4067-9a29-4557e54d8361 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:31.913309 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ee020d-5d2f-4067-9a29-4557e54d8361 None admin] PATH_INFO: `/v3/groups/1690f24ae1274d30b06a3c3e5c02cf6f` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:31.914719 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee020d-5d2f-4067-9a29-4557e54d8361 None admin] RBAC: Authorizing `identity:get_group(group_id=1690f24ae1274d30b06a3c3e5c02cf6f)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:31.916536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ee020d-5d2f-4067-9a29-4557e54d8361 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:31.918392 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 177/376] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 6 16:54:31 2022] GET /identity/v3/groups/1690f24ae1274d30b06a3c3e5c02cf6f => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:31.924218 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee020d-5d2f-4067-9a29-4557e54d8361 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:31.933945 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:32.020028 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e186d656-3427-4209-9d16-c7b825fe9eca None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:32.064071 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02885f81-5cd0-498d-a515-678225a44b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:32.065258 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02885f81-5cd0-498d-a515-678225a44b7b None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:32.065387 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02885f81-5cd0-498d-a515-678225a44b7b None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:32.065567 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02885f81-5cd0-498d-a515-678225a44b7b None admin] PATH_INFO: `/v3/projects/18f61d57ca574df5935e41a1653e029c` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:32.066779 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02885f81-5cd0-498d-a515-678225a44b7b None admin] RBAC: Authorizing `identity:get_project(project_id=18f61d57ca574df5935e41a1653e029c)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:32.068257 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02885f81-5cd0-498d-a515-678225a44b7b None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:32.070276 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 200/377] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:54:31 2022] GET /identity/v3/projects/18f61d57ca574df5935e41a1653e029c => generated 344 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:32.075932 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02885f81-5cd0-498d-a515-678225a44b7b None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:32.091480 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ee020d-5d2f-4067-9a29-4557e54d8361 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:32.123052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:32.169757 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f026c-44bb-46c7-8372-ab78a97aa80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:32.170433 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f026c-44bb-46c7-8372-ab78a97aa80b None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:32.170722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f026c-44bb-46c7-8372-ab78a97aa80b None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:32.171002 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65f026c-44bb-46c7-8372-ab78a97aa80b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:32.171881 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65f026c-44bb-46c7-8372-ab78a97aa80b None admin] RBAC: Adding query filter params (group.id=1690f24ae1274d30b06a3c3e5c02cf6f, role.id=c31c5ee7ed8646e48fa81e85d10fcaf2, scope.project.id=aa8007020ec543c1a6405283ddd21067) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:32.172203 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65f026c-44bb-46c7-8372-ab78a97aa80b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:32.173677 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65f026c-44bb-46c7-8372-ab78a97aa80b None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:32.184163 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02885f81-5cd0-498d-a515-678225a44b7b None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:32.190057 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 178/378] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Jul 6 16:54:31 2022] GET /identity/v3/role_assignments?group.id=1690f24ae1274d30b06a3c3e5c02cf6f&scope.project.id=aa8007020ec543c1a6405283ddd21067&role.id=c31c5ee7ed8646e48fa81e85d10fcaf2 => generated 260 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:32.223581 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb50d9ca-6e94-4e5b-b507-0ccecc88716c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:32.224509 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50d9ca-6e94-4e5b-b507-0ccecc88716c None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:32.224792 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50d9ca-6e94-4e5b-b507-0ccecc88716c None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:32.225066 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb50d9ca-6e94-4e5b-b507-0ccecc88716c None admin] PATH_INFO: `/v3/groups/cdac86b5299b4addb0e7207f9b5ac342` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:32.231453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb50d9ca-6e94-4e5b-b507-0ccecc88716c None admin] RBAC: Authorizing `identity:get_group(group_id=cdac86b5299b4addb0e7207f9b5ac342)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:32.233023 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb50d9ca-6e94-4e5b-b507-0ccecc88716c None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:32.235082 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 201/379] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 6 16:54:32 2022] GET /identity/v3/groups/cdac86b5299b4addb0e7207f9b5ac342 => generated 229 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:32.240721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f026c-44bb-46c7-8372-ab78a97aa80b None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:32.304172 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:32.373009 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65f026c-44bb-46c7-8372-ab78a97aa80b None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:32.413096 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e28c1b5-6087-41f6-af29-0759744e49a6 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:32.413418 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e28c1b5-6087-41f6-af29-0759744e49a6 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:32.413650 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e28c1b5-6087-41f6-af29-0759744e49a6 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:32.415160 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 202/380] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 16:54:32.416699 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb0992-8d0d-403c-907e-8e71658e61d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:32.417456 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb0992-8d0d-403c-907e-8e71658e61d7 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:32.417740 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb0992-8d0d-403c-907e-8e71658e61d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:32.418011 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb0992-8d0d-403c-907e-8e71658e61d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:32.418846 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fb0992-8d0d-403c-907e-8e71658e61d7 None admin] RBAC: Adding query filter params (group.id=cdac86b5299b4addb0e7207f9b5ac342, role.id=b41f8a7a58e64f749f2099ca45cc04a7, scope.project.id=18f61d57ca574df5935e41a1653e029c) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:32.419156 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fb0992-8d0d-403c-907e-8e71658e61d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:32.420626 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fb0992-8d0d-403c-907e-8e71658e61d7 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:32.422719 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591bcf0-05ea-4730-a2bc-62701879ff02 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:32.423014 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591bcf0-05ea-4730-a2bc-62701879ff02 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:32.423293 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591bcf0-05ea-4730-a2bc-62701879ff02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:32.429287 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 179/381] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Jul 6 16:54:32 2022] GET /identity/v3/role_assignments?group.id=cdac86b5299b4addb0e7207f9b5ac342&scope.project.id=18f61d57ca574df5935e41a1653e029c&role.id=b41f8a7a58e64f749f2099ca45cc04a7 => generated 260 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:32.455757 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-7591bcf0-05ea-4730-a2bc-62701879ff02 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:32.457020 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-7591bcf0-05ea-4730-a2bc-62701879ff02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:32.559133 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 203/382] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:32.594137 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb0992-8d0d-403c-907e-8e71658e61d7 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:32.746097 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb0992-8d0d-403c-907e-8e71658e61d7 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:32.752774 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:32.794279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46283bf-d591-4271-803d-2f7d2e423e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:32.795086 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46283bf-d591-4271-803d-2f7d2e423e02 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:32.795427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46283bf-d591-4271-803d-2f7d2e423e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:32.795706 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46283bf-d591-4271-803d-2f7d2e423e02 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:32.812319 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46283bf-d591-4271-803d-2f7d2e423e02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:32.820126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-d46283bf-d591-4271-803d-2f7d2e423e02 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 16:54:32.820126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:32.820126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:32.820126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:32.820126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:32.820126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:32.820126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:32.820126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:32.820126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:32.820126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:32.820126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:32.820126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:32.820126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:32.820126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:32.820126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:32.820126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:32.821130 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:32.821130 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:32.821130 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:32.821130 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:32.821130 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:32.821130 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:32.821130 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:32.821130 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:32.821130 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:32.821130 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:32.821130 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:32.821130 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:32.821130 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:32.821130 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:32.821130 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:32.821130 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:32.821949 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:32.821949 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:32.821949 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:32.821949 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:32.821949 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:32.821949 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:32.821949 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:32.821949 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:32.821949 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:32.821949 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:32.821949 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 16:54:32.821949 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:32.821949 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 180/383] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:54:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:32.827873 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7591bcf0-05ea-4730-a2bc-62701879ff02 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:32.879401 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:32.963759 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7591bcf0-05ea-4730-a2bc-62701879ff02 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:33.011243 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f4824-984e-445f-a1c2-11092ed2d9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:33.012301 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f4824-984e-445f-a1c2-11092ed2d9fc None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:33.012821 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f4824-984e-445f-a1c2-11092ed2d9fc None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:33.013316 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f4824-984e-445f-a1c2-11092ed2d9fc None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:33.014250 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3f4824-984e-445f-a1c2-11092ed2d9fc None admin] RBAC: Adding query filter params (name=member) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:33.014734 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3f4824-984e-445f-a1c2-11092ed2d9fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:33.016330 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3f4824-984e-445f-a1c2-11092ed2d9fc None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:33.026581 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 204/384] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 6 16:54:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:33.031835 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46283bf-d591-4271-803d-2f7d2e423e02 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:33.172464 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46283bf-d591-4271-803d-2f7d2e423e02 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:33.208892 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b21c3b8-7ae6-4dca-a905-7da45ddfdab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:33.209766 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21c3b8-7ae6-4dca-a905-7da45ddfdab4 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:33.210052 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21c3b8-7ae6-4dca-a905-7da45ddfdab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:33.210332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21c3b8-7ae6-4dca-a905-7da45ddfdab4 None admin] PATH_INFO: `/v3/groups/1690f24ae1274d30b06a3c3e5c02cf6f` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:33.212537 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b21c3b8-7ae6-4dca-a905-7da45ddfdab4 None admin] RBAC: Authorizing `identity:get_group(group_id=1690f24ae1274d30b06a3c3e5c02cf6f)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:33.214571 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b21c3b8-7ae6-4dca-a905-7da45ddfdab4 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:33.217322 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 181/385] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 6 16:54:33 2022] GET /identity/v3/groups/1690f24ae1274d30b06a3c3e5c02cf6f => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:33.217567 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438959e-093f-4686-92f5-3a97b4d7b7e7 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:33.217984 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438959e-093f-4686-92f5-3a97b4d7b7e7 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:33.218144 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438959e-093f-4686-92f5-3a97b4d7b7e7 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:33.219806 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 205/386] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 16:54:33.221444 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b21c3b8-7ae6-4dca-a905-7da45ddfdab4 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:33.226604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578b6e1-3558-4328-a2af-47b2a2d92eda None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:33.227160 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578b6e1-3558-4328-a2af-47b2a2d92eda None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:33.227657 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7578b6e1-3558-4328-a2af-47b2a2d92eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:33.254635 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-7578b6e1-3558-4328-a2af-47b2a2d92eda None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:33.256958 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-7578b6e1-3558-4328-a2af-47b2a2d92eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:33.284176 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:33.380689 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b21c3b8-7ae6-4dca-a905-7da45ddfdab4 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:33.422598 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c32a7-db30-46c7-a8ab-dfa55050c238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:33.424982 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c32a7-db30-46c7-a8ab-dfa55050c238 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:33.425822 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c32a7-db30-46c7-a8ab-dfa55050c238 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:33.426814 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3c32a7-db30-46c7-a8ab-dfa55050c238 None admin] PATH_INFO: `/v3/projects/aa8007020ec543c1a6405283ddd21067` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:33.428291 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 206/387] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:33.432451 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3c32a7-db30-46c7-a8ab-dfa55050c238 None admin] RBAC: Authorizing `identity:get_project(project_id=aa8007020ec543c1a6405283ddd21067)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:33.434334 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3c32a7-db30-46c7-a8ab-dfa55050c238 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:33.436513 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 182/388] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:54:33 2022] GET /identity/v3/projects/aa8007020ec543c1a6405283ddd21067 => generated 302 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:33.441644 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7578b6e1-3558-4328-a2af-47b2a2d92eda None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:33.453359 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c32a7-db30-46c7-a8ab-dfa55050c238 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:33.482659 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:33.560725 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7578b6e1-3558-4328-a2af-47b2a2d92eda None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:33.564408 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:33.568051 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3c32a7-db30-46c7-a8ab-dfa55050c238 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:33.599761 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1192b28d-edeb-4069-a131-bc988e60e823 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:33.599761 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192b28d-edeb-4069-a131-bc988e60e823 None admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:33.599761 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192b28d-edeb-4069-a131-bc988e60e823 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:33.599761 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1192b28d-edeb-4069-a131-bc988e60e823 None admin] PATH_INFO: `/v3/projects/aa8007020ec543c1a6405283ddd21067/groups/1690f24ae1274d30b06a3c3e5c02cf6f/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:33.600194 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1192b28d-edeb-4069-a131-bc988e60e823 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa8007020ec543c1a6405283ddd21067, group_id=1690f24ae1274d30b06a3c3e5c02cf6f, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:33.602758 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1192b28d-edeb-4069-a131-bc988e60e823 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:33.615397 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5e442-2259-4388-b6d2-971a372652cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:33.616313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5e442-2259-4388-b6d2-971a372652cc None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:33.619489 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5e442-2259-4388-b6d2-971a372652cc None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:33.619489 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c5e442-2259-4388-b6d2-971a372652cc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:33.630600 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c5e442-2259-4388-b6d2-971a372652cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:33.643482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-79c5e442-2259-4388-b6d2-971a372652cc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 16:54:33.643482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:33.643482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:33.643482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:33.643482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:33.643482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:33.643482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:33.643482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:33.643482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:33.643482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:33.643482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:33.643482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:33.643482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:33.643482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:33.643482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:33.643482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:33.644454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:33.644454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:33.644454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:33.644454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:33.644454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:33.644454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:33.644454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:33.644454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:33.644454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:33.644454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:33.644454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:33.644454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:33.644454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:33.644454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:33.644454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:33.644454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:33.645275 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:33.645275 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:33.645275 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:33.645275 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:33.645275 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:33.645275 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:33.645275 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:33.645275 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:33.645275 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:33.645275 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:33.645275 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 16:54:33.645275 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:33.645275 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 183/389] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:54:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:54:33.651977 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5e442-2259-4388-b6d2-971a372652cc None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:33.738559 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 207/390] 10.222.0.40 () {58 vars in 1442 bytes} [Wed Jul 6 16:54:33 2022] PUT /identity/v3/projects/aa8007020ec543c1a6405283ddd21067/groups/1690f24ae1274d30b06a3c3e5c02cf6f/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 16:54:33.765552 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c5e442-2259-4388-b6d2-971a372652cc None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:33.803288 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fbef93-7fd7-4e20-8b80-d66ec8c6a8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:33.804026 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fbef93-7fd7-4e20-8b80-d66ec8c6a8de None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:33.804315 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fbef93-7fd7-4e20-8b80-d66ec8c6a8de None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:33.804593 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fbef93-7fd7-4e20-8b80-d66ec8c6a8de None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:33.805364 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fbef93-7fd7-4e20-8b80-d66ec8c6a8de None admin] RBAC: Adding query filter params (name=member) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:33.805675 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fbef93-7fd7-4e20-8b80-d66ec8c6a8de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:33.807617 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fbef93-7fd7-4e20-8b80-d66ec8c6a8de None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:33.817666 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 184/391] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 6 16:54:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:33.823934 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1192b28d-edeb-4069-a131-bc988e60e823 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:33.945481 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:33.962869 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1192b28d-edeb-4069-a131-bc988e60e823 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:34.000360 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40db559-84f5-472f-88f7-0bb1406d1aa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:34.001321 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40db559-84f5-472f-88f7-0bb1406d1aa2 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:34.001612 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40db559-84f5-472f-88f7-0bb1406d1aa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:34.001894 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40db559-84f5-472f-88f7-0bb1406d1aa2 None admin] PATH_INFO: `/v3/groups/1690f24ae1274d30b06a3c3e5c02cf6f` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:34.003232 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40db559-84f5-472f-88f7-0bb1406d1aa2 None admin] RBAC: Authorizing `identity:get_group(group_id=1690f24ae1274d30b06a3c3e5c02cf6f)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:34.004734 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40db559-84f5-472f-88f7-0bb1406d1aa2 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:34.010418 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 208/392] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 6 16:54:33 2022] GET /identity/v3/groups/1690f24ae1274d30b06a3c3e5c02cf6f => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:34.012554 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fbef93-7fd7-4e20-8b80-d66ec8c6a8de None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:34.040274 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:34.118933 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fbef93-7fd7-4e20-8b80-d66ec8c6a8de None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:34.154766 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e71b2f5-21dc-4f3c-8faf-828178d235d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:34.155631 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71b2f5-21dc-4f3c-8faf-828178d235d5 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:34.155948 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71b2f5-21dc-4f3c-8faf-828178d235d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:34.156227 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e71b2f5-21dc-4f3c-8faf-828178d235d5 None admin] PATH_INFO: `/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:34.157473 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e71b2f5-21dc-4f3c-8faf-828178d235d5 None admin] RBAC: Authorizing `identity:get_project(project_id=49b1ed5ea8fc417f8247e689e804e6f1)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:34.158969 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e71b2f5-21dc-4f3c-8faf-828178d235d5 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:34.160986 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 185/393] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:54:34 2022] GET /identity/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:34.167225 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40db559-84f5-472f-88f7-0bb1406d1aa2 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:34.292376 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40db559-84f5-472f-88f7-0bb1406d1aa2 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:34.297439 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d9cc7-908f-45f8-ad2b-93581cae96e9 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:34.297979 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d9cc7-908f-45f8-ad2b-93581cae96e9 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:34.298451 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6d9cc7-908f-45f8-ad2b-93581cae96e9 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:34.300069 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 186/394] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 16:54:34.306134 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea75e3a-2ac1-4d7e-944d-e3cb0f3364af None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:34.306685 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea75e3a-2ac1-4d7e-944d-e3cb0f3364af None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:34.307569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea75e3a-2ac1-4d7e-944d-e3cb0f3364af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:34.343294 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-dea75e3a-2ac1-4d7e-944d-e3cb0f3364af None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:34.345910 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-dea75e3a-2ac1-4d7e-944d-e3cb0f3364af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:34.358496 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d867c-88ef-4533-a8cf-12c8c8b33715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:34.359646 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d867c-88ef-4533-a8cf-12c8c8b33715 None admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:34.360145 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d867c-88ef-4533-a8cf-12c8c8b33715 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:34.361099 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d867c-88ef-4533-a8cf-12c8c8b33715 None admin] PATH_INFO: `/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1/groups/1690f24ae1274d30b06a3c3e5c02cf6f/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:34.366494 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47d867c-88ef-4533-a8cf-12c8c8b33715 None admin] RBAC: Authorizing `identity:create_grant(project_id=49b1ed5ea8fc417f8247e689e804e6f1, group_id=1690f24ae1274d30b06a3c3e5c02cf6f, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:34.368200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47d867c-88ef-4533-a8cf-12c8c8b33715 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:34.432586 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 187/395] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 16:54:34.455378 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea75e3a-2ac1-4d7e-944d-e3cb0f3364af None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:34.468087 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 209/396] 10.222.0.40 () {58 vars in 1442 bytes} [Wed Jul 6 16:54:34 2022] PUT /identity/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1/groups/1690f24ae1274d30b06a3c3e5c02cf6f/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 16:54:34.483794 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:34.595439 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea75e3a-2ac1-4d7e-944d-e3cb0f3364af None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:34.640728 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676a58d6-b448-43e8-a58e-8a2c49eef8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:34.641544 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a58d6-b448-43e8-a58e-8a2c49eef8c5 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:34.641824 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a58d6-b448-43e8-a58e-8a2c49eef8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:34.642098 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a58d6-b448-43e8-a58e-8a2c49eef8c5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:34.653289 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676a58d6-b448-43e8-a58e-8a2c49eef8c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:34.661609 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-676a58d6-b448-43e8-a58e-8a2c49eef8c5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 16:54:34.661609 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:34.661609 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:34.661609 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:34.661609 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:34.661609 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:34.661609 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:34.661609 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:34.661609 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:34.661609 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:34.661609 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:34.661609 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:34.661609 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:34.661609 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:34.661609 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:34.661609 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:34.662621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:34.662621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:34.662621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:34.662621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:34.662621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:34.662621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:34.662621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:34.662621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:34.662621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:34.662621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:34.662621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:34.662621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:34.662621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:34.662621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:34.662621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:34.662621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:34.663415 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:34.663415 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:34.663415 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:34.663415 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:34.663415 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:34.663415 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:34.663415 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:34.663415 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:34.663415 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:34.663415 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:34.663415 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 16:54:34.663415 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:34.663415 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 188/397] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 6 16:54:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:54:34.669640 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d867c-88ef-4533-a8cf-12c8c8b33715 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:34.694511 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:34.776997 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d867c-88ef-4533-a8cf-12c8c8b33715 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:34.814519 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d41f11-081a-4ad2-8ea0-5f04453f290d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:34.815207 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d41f11-081a-4ad2-8ea0-5f04453f290d None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:34.815490 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d41f11-081a-4ad2-8ea0-5f04453f290d None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:34.815797 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d41f11-081a-4ad2-8ea0-5f04453f290d None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:34.816551 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d41f11-081a-4ad2-8ea0-5f04453f290d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:34.816910 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d41f11-081a-4ad2-8ea0-5f04453f290d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:34.818293 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d41f11-081a-4ad2-8ea0-5f04453f290d None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:34.827045 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 210/398] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:54:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:34.832529 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676a58d6-b448-43e8-a58e-8a2c49eef8c5 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:34.857910 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12b3cb-4f32-4741-8ed1-2bd7b0589140 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:34.858221 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12b3cb-4f32-4741-8ed1-2bd7b0589140 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:34.858525 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12b3cb-4f32-4741-8ed1-2bd7b0589140 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:34.860210 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 211/399] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 16:54:34.865866 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb85cf-1738-455f-86a8-7ce84942a65f None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:34.866145 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb85cf-1738-455f-86a8-7ce84942a65f None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:34.866416 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0eb85cf-1738-455f-86a8-7ce84942a65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:34.885845 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-c0eb85cf-1738-455f-86a8-7ce84942a65f None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:34.887028 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c0eb85cf-1738-455f-86a8-7ce84942a65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:34.941707 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:34.961921 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676a58d6-b448-43e8-a58e-8a2c49eef8c5 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:34.965034 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 212/400] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 16:54:34.966541 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e81a8-3ab1-4fc9-922a-9e765f892952 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:34.966834 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e81a8-3ab1-4fc9-922a-9e765f892952 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:34.967118 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e81a8-3ab1-4fc9-922a-9e765f892952 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:34.972431 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 213/401] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:34.973914 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e64068-189a-44b5-a0db-92de727b861d None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:34.974209 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e64068-189a-44b5-a0db-92de727b861d None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:34.974497 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e64068-189a-44b5-a0db-92de727b861d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:34.993087 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-b6e64068-189a-44b5-a0db-92de727b861d None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:34.994838 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b6e64068-189a-44b5-a0db-92de727b861d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:34.997364 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:35.020200 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4b32b-abd3-416b-8c7f-bd47c5c1886f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:35.021699 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4b32b-abd3-416b-8c7f-bd47c5c1886f None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:35.022199 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4b32b-abd3-416b-8c7f-bd47c5c1886f None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:35.022691 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f4b32b-abd3-416b-8c7f-bd47c5c1886f None admin] PATH_INFO: `/v3/groups/1690f24ae1274d30b06a3c3e5c02cf6f` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:35.024038 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f4b32b-abd3-416b-8c7f-bd47c5c1886f None admin] RBAC: Authorizing `identity:get_group(group_id=1690f24ae1274d30b06a3c3e5c02cf6f)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:35.026264 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f4b32b-abd3-416b-8c7f-bd47c5c1886f None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:35.028193 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 189/402] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 6 16:54:34 2022] GET /identity/v3/groups/1690f24ae1274d30b06a3c3e5c02cf6f => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:35.029277 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4b32b-abd3-416b-8c7f-bd47c5c1886f None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:35.084163 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 214/403] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:35.088069 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e64068-189a-44b5-a0db-92de727b861d None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:35.153033 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f4b32b-abd3-416b-8c7f-bd47c5c1886f None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:35.189901 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c71bf9-fc0f-4e38-9273-8fc477e572ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:35.190973 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c71bf9-fc0f-4e38-9273-8fc477e572ae None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:35.191401 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c71bf9-fc0f-4e38-9273-8fc477e572ae None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:35.191858 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c71bf9-fc0f-4e38-9273-8fc477e572ae None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:35.202653 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c71bf9-fc0f-4e38-9273-8fc477e572ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:35.211204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-67c71bf9-fc0f-4e38-9273-8fc477e572ae None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 16:54:35.211204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:35.211204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:35.211204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:35.211204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:35.211204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:35.211204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:35.211204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:35.211204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:35.211204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:35.211204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:35.211204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:35.211204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:35.211204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:35.211204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:35.211204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:35.212236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:35.212236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:35.212236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:35.212236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:35.212236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:35.212236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:35.212236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:35.212236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:35.212236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:35.212236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:35.212236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:35.212236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:35.212236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:35.212236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:35.212236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:35.212236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:35.213605 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:35.213605 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:35.213605 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:35.213605 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:35.213605 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:35.213605 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:35.213605 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:35.213605 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:35.213605 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:35.213605 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:35.213605 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 16:54:35.213605 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:35.216044 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 190/404] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 6 16:54:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:35.221609 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c71bf9-fc0f-4e38-9273-8fc477e572ae None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:35.222870 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e64068-189a-44b5-a0db-92de727b861d None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:35.235496 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:35.270416 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bcbef4-965f-49cc-9879-9cb146909f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:35.271716 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bcbef4-965f-49cc-9879-9cb146909f86 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:35.272033 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bcbef4-965f-49cc-9879-9cb146909f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:35.272325 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bcbef4-965f-49cc-9879-9cb146909f86 None admin] PATH_INFO: `/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:35.274168 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bcbef4-965f-49cc-9879-9cb146909f86 None admin] RBAC: Authorizing `identity:get_project(project_id=49b1ed5ea8fc417f8247e689e804e6f1)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:35.276411 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bcbef4-965f-49cc-9879-9cb146909f86 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:35.278794 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 215/405] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:54:35 2022] GET /identity/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:35.279689 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bcbef4-965f-49cc-9879-9cb146909f86 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:35.296055 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:35.331593 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c71bf9-fc0f-4e38-9273-8fc477e572ae None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:35.369681 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cf0c21-1d50-48bb-9533-7f418abc71ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:35.370912 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf0c21-1d50-48bb-9533-7f418abc71ca None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:35.371396 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf0c21-1d50-48bb-9533-7f418abc71ca None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:35.371920 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf0c21-1d50-48bb-9533-7f418abc71ca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:35.383338 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cf0c21-1d50-48bb-9533-7f418abc71ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:35.392091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-32cf0c21-1d50-48bb-9533-7f418abc71ca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:54:35.392091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:35.392091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:35.392091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:35.392091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:35.392091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:35.392091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:35.392091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:35.392091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:35.392091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:35.392091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:35.392091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:35.392091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:35.392091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:35.392091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:35.392091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:35.392984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:35.392984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:35.392984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:35.392984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:35.392984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:35.392984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:35.392984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:35.392984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:35.392984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:35.392984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:35.392984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:35.392984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:35.392984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:35.392984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:35.392984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:35.392984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:35.393790 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:35.393790 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:35.393790 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:35.393790 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:35.393790 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:35.393790 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:35.393790 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:35.393790 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:35.393790 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:35.393790 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:35.393790 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:54:35.393790 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:35.398821 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 191/406] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 6 16:54:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:35.399851 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cf0c21-1d50-48bb-9533-7f418abc71ca None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:35.419902 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bcbef4-965f-49cc-9879-9cb146909f86 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:35.456427 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c367f8b-e6e4-45e5-b3ad-5d1d0b580ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:35.456995 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c367f8b-e6e4-45e5-b3ad-5d1d0b580ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:35.457353 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c367f8b-e6e4-45e5-b3ad-5d1d0b580ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:35.457698 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c367f8b-e6e4-45e5-b3ad-5d1d0b580ae7 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:35.458514 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c367f8b-e6e4-45e5-b3ad-5d1d0b580ae7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:35.458905 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c367f8b-e6e4-45e5-b3ad-5d1d0b580ae7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:35.460567 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c367f8b-e6e4-45e5-b3ad-5d1d0b580ae7 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:35.474602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 216/407] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:54:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:35.475600 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c367f8b-e6e4-45e5-b3ad-5d1d0b580ae7 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:35.489924 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:35.537664 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cf0c21-1d50-48bb-9533-7f418abc71ca None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:35.550505 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:35.584606 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01453971-53e1-492a-9e5e-5842d07132f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:35.586109 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01453971-53e1-492a-9e5e-5842d07132f6 None admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:35.586473 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01453971-53e1-492a-9e5e-5842d07132f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:35.586826 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01453971-53e1-492a-9e5e-5842d07132f6 None admin] PATH_INFO: `/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1/groups/1690f24ae1274d30b06a3c3e5c02cf6f/roles/c31c5ee7ed8646e48fa81e85d10fcaf2` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:35.588742 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c367f8b-e6e4-45e5-b3ad-5d1d0b580ae7 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:35.589924 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01453971-53e1-492a-9e5e-5842d07132f6 None admin] RBAC: Authorizing `identity:create_grant(project_id=49b1ed5ea8fc417f8247e689e804e6f1, group_id=1690f24ae1274d30b06a3c3e5c02cf6f, role_id=c31c5ee7ed8646e48fa81e85d10fcaf2)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:35.592237 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01453971-53e1-492a-9e5e-5842d07132f6 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:35.629342 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39468f62-9828-46f7-b0c8-b2bf8b1f3829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:35.630018 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39468f62-9828-46f7-b0c8-b2bf8b1f3829 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:35.630308 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39468f62-9828-46f7-b0c8-b2bf8b1f3829 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:35.630584 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39468f62-9828-46f7-b0c8-b2bf8b1f3829 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:35.631338 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39468f62-9828-46f7-b0c8-b2bf8b1f3829 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:35.631653 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39468f62-9828-46f7-b0c8-b2bf8b1f3829 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:35.633257 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39468f62-9828-46f7-b0c8-b2bf8b1f3829 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:35.647411 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 217/408] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:54:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:35.647411 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39468f62-9828-46f7-b0c8-b2bf8b1f3829 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:35.684136 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 192/409] 10.222.0.40 () {58 vars in 1442 bytes} [Wed Jul 6 16:54:35 2022] PUT /identity/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1/groups/1690f24ae1274d30b06a3c3e5c02cf6f/roles/c31c5ee7ed8646e48fa81e85d10fcaf2 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 16:54:35.685967 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01453971-53e1-492a-9e5e-5842d07132f6 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:35.841120 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39468f62-9828-46f7-b0c8-b2bf8b1f3829 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:35.891548 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:35.898486 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c06564-1989-42c9-b1e1-57b2ca5a2ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:35.898669 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01453971-53e1-492a-9e5e-5842d07132f6 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:35.899358 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c06564-1989-42c9-b1e1-57b2ca5a2ba6 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:35.899643 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c06564-1989-42c9-b1e1-57b2ca5a2ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:35.900600 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c06564-1989-42c9-b1e1-57b2ca5a2ba6 None admin] PATH_INFO: `/v3/groups/1690f24ae1274d30b06a3c3e5c02cf6f` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:35.907380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c06564-1989-42c9-b1e1-57b2ca5a2ba6 None admin] RBAC: Authorizing `identity:get_group(group_id=1690f24ae1274d30b06a3c3e5c02cf6f)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:35.907380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c06564-1989-42c9-b1e1-57b2ca5a2ba6 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:35.907380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 218/410] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 6 16:54:35 2022] GET /identity/v3/groups/1690f24ae1274d30b06a3c3e5c02cf6f => generated 226 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:35.910778 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c06564-1989-42c9-b1e1-57b2ca5a2ba6 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:35.928147 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:35.936317 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d86a0b-247b-4d61-b87d-a50661454922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:35.937199 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d86a0b-247b-4d61-b87d-a50661454922 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:35.937530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d86a0b-247b-4d61-b87d-a50661454922 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:35.937801 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d86a0b-247b-4d61-b87d-a50661454922 None admin] PATH_INFO: `/v3/groups/cdac86b5299b4addb0e7207f9b5ac342` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:35.939161 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d86a0b-247b-4d61-b87d-a50661454922 None admin] RBAC: Authorizing `identity:get_group(group_id=cdac86b5299b4addb0e7207f9b5ac342)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:35.940682 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d86a0b-247b-4d61-b87d-a50661454922 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:35.942836 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 193/411] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 6 16:54:35 2022] GET /identity/v3/groups/cdac86b5299b4addb0e7207f9b5ac342 => generated 229 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:35.948243 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d86a0b-247b-4d61-b87d-a50661454922 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:36.047601 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c06564-1989-42c9-b1e1-57b2ca5a2ba6 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:36.102003 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d86a0b-247b-4d61-b87d-a50661454922 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:36.105909 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4d8de8-9880-4684-afd1-ec52811e8fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:36.106781 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4d8de8-9880-4684-afd1-ec52811e8fc8 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:36.107058 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4d8de8-9880-4684-afd1-ec52811e8fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:36.107330 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4d8de8-9880-4684-afd1-ec52811e8fc8 None admin] PATH_INFO: `/v3/projects/aa8007020ec543c1a6405283ddd21067` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:36.109278 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4d8de8-9880-4684-afd1-ec52811e8fc8 None admin] RBAC: Authorizing `identity:get_project(project_id=aa8007020ec543c1a6405283ddd21067)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:36.110698 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4d8de8-9880-4684-afd1-ec52811e8fc8 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:36.112668 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 219/412] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:54:35 2022] GET /identity/v3/projects/aa8007020ec543c1a6405283ddd21067 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:36.117776 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4d8de8-9880-4684-afd1-ec52811e8fc8 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:36.167963 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0638fd5e-e75f-4ab4-b5c1-0b829477674d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:36.168884 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638fd5e-e75f-4ab4-b5c1-0b829477674d None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:36.169167 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638fd5e-e75f-4ab4-b5c1-0b829477674d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:36.169438 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0638fd5e-e75f-4ab4-b5c1-0b829477674d None admin] PATH_INFO: `/v3/projects/18f61d57ca574df5935e41a1653e029c` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:36.170718 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0638fd5e-e75f-4ab4-b5c1-0b829477674d None admin] RBAC: Authorizing `identity:get_project(project_id=18f61d57ca574df5935e41a1653e029c)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:36.172176 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0638fd5e-e75f-4ab4-b5c1-0b829477674d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:36.175614 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 194/413] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:54:35 2022] GET /identity/v3/projects/18f61d57ca574df5935e41a1653e029c => generated 344 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:36.180454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0638fd5e-e75f-4ab4-b5c1-0b829477674d None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:36.235122 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:36.273631 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:36.317562 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0638fd5e-e75f-4ab4-b5c1-0b829477674d None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:36.329842 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4d8de8-9880-4684-afd1-ec52811e8fc8 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:36.358174 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b771c33a-94ff-48d1-95e1-38706fd2e9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:36.359229 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771c33a-94ff-48d1-95e1-38706fd2e9ea None admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:36.359844 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771c33a-94ff-48d1-95e1-38706fd2e9ea None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:36.360136 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b771c33a-94ff-48d1-95e1-38706fd2e9ea None admin] PATH_INFO: `/v3/projects/18f61d57ca574df5935e41a1653e029c/groups/cdac86b5299b4addb0e7207f9b5ac342/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:36.362815 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b771c33a-94ff-48d1-95e1-38706fd2e9ea None admin] RBAC: Authorizing `identity:create_grant(project_id=18f61d57ca574df5935e41a1653e029c, group_id=cdac86b5299b4addb0e7207f9b5ac342, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:36.364552 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b771c33a-94ff-48d1-95e1-38706fd2e9ea None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:36.371009 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1a7ece-1478-476e-868e-a2c1a5fe2848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:36.371986 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a7ece-1478-476e-868e-a2c1a5fe2848 None admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:36.372273 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a7ece-1478-476e-868e-a2c1a5fe2848 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:36.372550 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1a7ece-1478-476e-868e-a2c1a5fe2848 None admin] PATH_INFO: `/v3/projects/aa8007020ec543c1a6405283ddd21067/groups/1690f24ae1274d30b06a3c3e5c02cf6f/roles/c31c5ee7ed8646e48fa81e85d10fcaf2` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:36.374865 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1a7ece-1478-476e-868e-a2c1a5fe2848 None admin] RBAC: Authorizing `identity:create_grant(project_id=aa8007020ec543c1a6405283ddd21067, group_id=1690f24ae1274d30b06a3c3e5c02cf6f, role_id=c31c5ee7ed8646e48fa81e85d10fcaf2)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:36.376343 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1a7ece-1478-476e-868e-a2c1a5fe2848 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:36.450899 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 195/414] 10.222.0.40 () {58 vars in 1442 bytes} [Wed Jul 6 16:54:36 2022] PUT /identity/v3/projects/18f61d57ca574df5935e41a1653e029c/groups/cdac86b5299b4addb0e7207f9b5ac342/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 16:54:36.450899 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b849be-c039-4b1a-97af-954d684dd59b None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:36.450899 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b849be-c039-4b1a-97af-954d684dd59b None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:36.450899 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b849be-c039-4b1a-97af-954d684dd59b None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:36.451404 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 196/415] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:36.457369 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff3875-5513-4ffe-9005-5c3d264d2856 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:36.457684 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff3875-5513-4ffe-9005-5c3d264d2856 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:36.457950 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ff3875-5513-4ffe-9005-5c3d264d2856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:36.475950 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 220/416] 10.222.0.40 () {58 vars in 1442 bytes} [Wed Jul 6 16:54:36 2022] PUT /identity/v3/projects/aa8007020ec543c1a6405283ddd21067/groups/1690f24ae1274d30b06a3c3e5c02cf6f/roles/c31c5ee7ed8646e48fa81e85d10fcaf2 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 16:54:36.484501 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-13ff3875-5513-4ffe-9005-5c3d264d2856 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:36.485785 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-13ff3875-5513-4ffe-9005-5c3d264d2856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:36.574168 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 197/417] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:36.611174 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1a7ece-1478-476e-868e-a2c1a5fe2848 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:36.747591 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1a7ece-1478-476e-868e-a2c1a5fe2848 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:36.775568 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:36.785314 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57146ab-a65a-4680-87b5-c37849b39d54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:36.793898 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57146ab-a65a-4680-87b5-c37849b39d54 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:36.793898 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57146ab-a65a-4680-87b5-c37849b39d54 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:36.793898 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57146ab-a65a-4680-87b5-c37849b39d54 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:36.797645 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57146ab-a65a-4680-87b5-c37849b39d54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:36.805320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-f57146ab-a65a-4680-87b5-c37849b39d54 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 16:54:36.805320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:36.805320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:36.805320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:36.805320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:36.805320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:36.805320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:36.805320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:36.805320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:36.805320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:36.805320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:36.805320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:36.805320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:36.805320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:36.805320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:36.805320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:36.806778 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:36.806778 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:36.806778 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:36.806778 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:36.806778 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:36.806778 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:36.806778 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:36.806778 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:36.806778 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:36.806778 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:36.806778 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:36.806778 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:36.806778 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:36.806778 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:36.806778 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:36.806778 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:36.808199 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:36.808199 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:36.808199 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:36.808199 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:36.808199 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:36.808199 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:36.808199 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:36.808199 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:36.808199 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:36.808199 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:36.808199 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 16:54:36.808199 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:36.808199 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 221/418] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:54:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:36.812776 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff3875-5513-4ffe-9005-5c3d264d2856 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:36.872098 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:36.920587 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ff3875-5513-4ffe-9005-5c3d264d2856 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:36.940512 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62211032-fc93-49e0-8fc9-65e77e7cbf8a None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:36.940826 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62211032-fc93-49e0-8fc9-65e77e7cbf8a None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:36.941075 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62211032-fc93-49e0-8fc9-65e77e7cbf8a None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:36.942717 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 222/419] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 16:54:36.947595 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d06875-10f1-442c-aa2c-ca9b6c6bf902 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:36.948127 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d06875-10f1-442c-aa2c-ca9b6c6bf902 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:36.948418 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d06875-10f1-442c-aa2c-ca9b6c6bf902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:36.956651 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a778e1-b953-4495-acac-9174fef73367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:36.957325 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a778e1-b953-4495-acac-9174fef73367 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:36.957609 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a778e1-b953-4495-acac-9174fef73367 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:36.957885 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a778e1-b953-4495-acac-9174fef73367 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:36.958641 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a778e1-b953-4495-acac-9174fef73367 None admin] RBAC: Adding query filter params (name=member) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:36.958944 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a778e1-b953-4495-acac-9174fef73367 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:36.960352 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a778e1-b953-4495-acac-9174fef73367 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:36.965722 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-f7d06875-10f1-442c-aa2c-ca9b6c6bf902 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:36.966923 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f7d06875-10f1-442c-aa2c-ca9b6c6bf902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:36.969194 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 198/420] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 6 16:54:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:36.974276 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a778e1-b953-4495-acac-9174fef73367 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:37.079332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a778e1-b953-4495-acac-9174fef73367 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:37.096447 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:37.102059 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 223/421] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 16:54:37.115352 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde1f341-8f43-43f1-8412-b02da7150f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:37.116047 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1f341-8f43-43f1-8412-b02da7150f49 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:37.116337 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1f341-8f43-43f1-8412-b02da7150f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:37.116614 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1f341-8f43-43f1-8412-b02da7150f49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:37.141464 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d06875-10f1-442c-aa2c-ca9b6c6bf902 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:37.195966 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde1f341-8f43-43f1-8412-b02da7150f49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:37.197564 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde1f341-8f43-43f1-8412-b02da7150f49 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:37.322550 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d06875-10f1-442c-aa2c-ca9b6c6bf902 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:37.357067 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:37.366517 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 199/422] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:54:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 16:54:37.371582 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde1f341-8f43-43f1-8412-b02da7150f49 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:37.391374 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14d9263-714d-434e-bbce-842b59c89a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:37.392266 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d9263-714d-434e-bbce-842b59c89a98 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:37.392497 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d9263-714d-434e-bbce-842b59c89a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:37.392774 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14d9263-714d-434e-bbce-842b59c89a98 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:37.399784 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:37.407755 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14d9263-714d-434e-bbce-842b59c89a98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:37.419539 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-e14d9263-714d-434e-bbce-842b59c89a98 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 06 16:54:37.419539 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:37.419539 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:37.419539 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:37.419539 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:37.419539 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:37.419539 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:37.419539 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:37.419539 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:37.419539 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:37.419539 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:37.419539 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:37.419539 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:37.419539 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:37.419539 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:37.419539 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:37.420446 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:37.420446 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:37.420446 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:37.420446 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:37.420446 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:37.420446 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:37.420446 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:37.420446 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:37.420446 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:37.420446 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:37.420446 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:37.420446 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:37.420446 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:37.420446 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:37.420446 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:37.420446 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:37.421253 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:37.421253 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:37.421253 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:37.421253 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:37.421253 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:37.421253 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:37.421253 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:37.421253 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:37.421253 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:37.421253 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:37.421253 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 06 16:54:37.421253 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:37.421253 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 224/423] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:54:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:54:37.428703 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14d9263-714d-434e-bbce-842b59c89a98 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:37.542055 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde1f341-8f43-43f1-8412-b02da7150f49 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:37.580952 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ee3af-6ba2-41ba-96ae-eff936207599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:37.581826 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ee3af-6ba2-41ba-96ae-eff936207599 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:37.582105 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ee3af-6ba2-41ba-96ae-eff936207599 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:37.582378 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ee3af-6ba2-41ba-96ae-eff936207599 None admin] PATH_INFO: `/v3/projects/aa8007020ec543c1a6405283ddd21067` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:37.583565 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366ee3af-6ba2-41ba-96ae-eff936207599 None admin] RBAC: Authorizing `identity:get_project(project_id=aa8007020ec543c1a6405283ddd21067)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:37.585184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366ee3af-6ba2-41ba-96ae-eff936207599 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:37.587241 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 200/424] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:54:37 2022] GET /identity/v3/projects/aa8007020ec543c1a6405283ddd21067 => generated 302 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:37.593199 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ee3af-6ba2-41ba-96ae-eff936207599 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:37.692221 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:37.702735 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ee3af-6ba2-41ba-96ae-eff936207599 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:37.723254 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14d9263-714d-434e-bbce-842b59c89a98 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:37.770093 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b0eac4-7fc6-4e31-8128-a619d3728696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:37.770964 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b0eac4-7fc6-4e31-8128-a619d3728696 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:37.771508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b0eac4-7fc6-4e31-8128-a619d3728696 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:37.771809 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b0eac4-7fc6-4e31-8128-a619d3728696 None admin] PATH_INFO: `/v3/groups/1690f24ae1274d30b06a3c3e5c02cf6f` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:37.774243 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b0eac4-7fc6-4e31-8128-a619d3728696 None admin] RBAC: Authorizing `identity:get_group(group_id=1690f24ae1274d30b06a3c3e5c02cf6f)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:37.775750 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b0eac4-7fc6-4e31-8128-a619d3728696 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:37.777577 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 201/425] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 6 16:54:37 2022] GET /identity/v3/groups/1690f24ae1274d30b06a3c3e5c02cf6f => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:37.783228 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b0eac4-7fc6-4e31-8128-a619d3728696 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:37.801040 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e4977-62d9-48bf-a2df-8ac3e4e61874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:37.801723 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e4977-62d9-48bf-a2df-8ac3e4e61874 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:37.802010 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e4977-62d9-48bf-a2df-8ac3e4e61874 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:37.802287 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e4977-62d9-48bf-a2df-8ac3e4e61874 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:37.803047 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4e4977-62d9-48bf-a2df-8ac3e4e61874 None admin] RBAC: Adding query filter params (name=member) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:37.803356 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4e4977-62d9-48bf-a2df-8ac3e4e61874 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:37.804784 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4e4977-62d9-48bf-a2df-8ac3e4e61874 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:37.819538 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 225/426] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 6 16:54:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 391 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:37.825758 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e4977-62d9-48bf-a2df-8ac3e4e61874 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:37.873595 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:37.936886 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4e4977-62d9-48bf-a2df-8ac3e4e61874 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:37.974184 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebff30ee-c5bf-4990-84ec-090d432083f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:37.975296 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff30ee-c5bf-4990-84ec-090d432083f1 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:37.975583 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff30ee-c5bf-4990-84ec-090d432083f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:37.975913 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebff30ee-c5bf-4990-84ec-090d432083f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:38.012184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b0eac4-7fc6-4e31-8128-a619d3728696 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:38.069437 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebff30ee-c5bf-4990-84ec-090d432083f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:38.070909 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebff30ee-c5bf-4990-84ec-090d432083f1 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:38.090716 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c74ce3-a14a-49a0-b74c-57f0165c278f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:38.091385 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c74ce3-a14a-49a0-b74c-57f0165c278f None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:38.091672 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c74ce3-a14a-49a0-b74c-57f0165c278f None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:38.091978 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c74ce3-a14a-49a0-b74c-57f0165c278f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:38.092823 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c74ce3-a14a-49a0-b74c-57f0165c278f None admin] RBAC: Adding query filter params (group.id=1690f24ae1274d30b06a3c3e5c02cf6f, role.id=7fd63d2d99254c00bb3501173643ee30, scope.project.id=aa8007020ec543c1a6405283ddd21067) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:38.093149 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c74ce3-a14a-49a0-b74c-57f0165c278f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:38.094586 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c74ce3-a14a-49a0-b74c-57f0165c278f None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:38.106330 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 202/427] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Jul 6 16:54:37 2022] GET /identity/v3/role_assignments?group.id=1690f24ae1274d30b06a3c3e5c02cf6f&scope.project.id=aa8007020ec543c1a6405283ddd21067&role.id=7fd63d2d99254c00bb3501173643ee30 => generated 611 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:38.108054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb37887-a85c-4277-8d13-572c22f5fa0e None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:38.108547 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb37887-a85c-4277-8d13-572c22f5fa0e None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:38.109031 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb37887-a85c-4277-8d13-572c22f5fa0e None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:38.110835 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 203/428] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:38.118215 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8fa8b6-f527-4889-882c-961cb66c900a None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:38.118855 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8fa8b6-f527-4889-882c-961cb66c900a None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:38.119181 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8fa8b6-f527-4889-882c-961cb66c900a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:38.135101 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:38.152307 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-bb8fa8b6-f527-4889-882c-961cb66c900a None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:38.153534 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-bb8fa8b6-f527-4889-882c-961cb66c900a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:38.181425 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:38.251989 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 226/429] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:54:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 16:54:38.257857 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebff30ee-c5bf-4990-84ec-090d432083f1 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:38.292159 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 204/430] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:38.316487 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8fa8b6-f527-4889-882c-961cb66c900a None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:38.389161 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebff30ee-c5bf-4990-84ec-090d432083f1 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:38.425199 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8fa8b6-f527-4889-882c-961cb66c900a None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:38.427217 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb46963-c44f-45ea-a1a8-5763007ce440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:38.428492 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb46963-c44f-45ea-a1a8-5763007ce440 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:38.429003 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb46963-c44f-45ea-a1a8-5763007ce440 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:38.430413 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb46963-c44f-45ea-a1a8-5763007ce440 None admin] PATH_INFO: `/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:38.431958 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb46963-c44f-45ea-a1a8-5763007ce440 None admin] RBAC: Authorizing `identity:get_project(project_id=49b1ed5ea8fc417f8247e689e804e6f1)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:38.433616 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb46963-c44f-45ea-a1a8-5763007ce440 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:38.436083 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 227/431] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:54:38 2022] GET /identity/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:38.444697 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb46963-c44f-45ea-a1a8-5763007ce440 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:38.454607 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:38.474512 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e030e1-9d8c-44e6-b984-97d43031260a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:38.475890 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e030e1-9d8c-44e6-b984-97d43031260a None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:38.476555 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e030e1-9d8c-44e6-b984-97d43031260a None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:38.476937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e030e1-9d8c-44e6-b984-97d43031260a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:38.495244 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e030e1-9d8c-44e6-b984-97d43031260a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:38.510106 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-75e030e1-9d8c-44e6-b984-97d43031260a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 16:54:38.510106 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:38.510106 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:38.510106 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:38.510106 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:38.510106 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:38.510106 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:38.510106 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:38.510106 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:38.510106 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:38.510106 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:38.510106 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:38.510106 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:38.510106 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:38.510106 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:38.510106 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:38.511054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:38.511054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:38.511054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:38.511054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:38.511054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:38.511054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:38.511054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:38.511054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:38.511054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:38.511054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:38.511054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:38.511054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:38.511054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:38.511054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:38.511054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:38.511054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:38.511885 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:38.511885 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:38.511885 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:38.511885 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:38.511885 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:38.511885 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:38.511885 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:38.511885 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:38.511885 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:38.511885 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:38.511885 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 16:54:38.511885 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:38.518158 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 205/432] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 6 16:54:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:54:38.518158 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e030e1-9d8c-44e6-b984-97d43031260a None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:38.531536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:38.603759 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb46963-c44f-45ea-a1a8-5763007ce440 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:38.640129 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71029dfb-b00e-4516-a509-a5395315b7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:38.640958 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71029dfb-b00e-4516-a509-a5395315b7be None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:38.641265 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71029dfb-b00e-4516-a509-a5395315b7be None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:38.641548 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71029dfb-b00e-4516-a509-a5395315b7be None admin] PATH_INFO: `/v3/groups/1690f24ae1274d30b06a3c3e5c02cf6f` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:38.642703 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71029dfb-b00e-4516-a509-a5395315b7be None admin] RBAC: Authorizing `identity:get_group(group_id=1690f24ae1274d30b06a3c3e5c02cf6f)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:38.644301 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71029dfb-b00e-4516-a509-a5395315b7be None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:38.646561 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 228/433] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 6 16:54:38 2022] GET /identity/v3/groups/1690f24ae1274d30b06a3c3e5c02cf6f => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:38.647638 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e030e1-9d8c-44e6-b984-97d43031260a None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:38.652376 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71029dfb-b00e-4516-a509-a5395315b7be None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:38.691763 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33033b3-bc4b-4577-9b29-5dc6bd90ba84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:38.692720 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33033b3-bc4b-4577-9b29-5dc6bd90ba84 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:38.693149 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33033b3-bc4b-4577-9b29-5dc6bd90ba84 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:38.693567 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33033b3-bc4b-4577-9b29-5dc6bd90ba84 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:38.694469 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33033b3-bc4b-4577-9b29-5dc6bd90ba84 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:38.694927 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33033b3-bc4b-4577-9b29-5dc6bd90ba84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:38.697600 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33033b3-bc4b-4577-9b29-5dc6bd90ba84 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:38.709348 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 206/434] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:54:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:38.716961 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33033b3-bc4b-4577-9b29-5dc6bd90ba84 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:38.751646 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:38.779354 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71029dfb-b00e-4516-a509-a5395315b7be None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:38.810655 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:38.818021 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0374a5e4-970c-457e-9165-66a1381c9cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:38.818948 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0374a5e4-970c-457e-9165-66a1381c9cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:38.819410 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0374a5e4-970c-457e-9165-66a1381c9cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:38.819898 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0374a5e4-970c-457e-9165-66a1381c9cf9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:38.821068 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0374a5e4-970c-457e-9165-66a1381c9cf9 None admin] RBAC: Adding query filter params (group.id=1690f24ae1274d30b06a3c3e5c02cf6f, role.id=7fd63d2d99254c00bb3501173643ee30, scope.project.id=49b1ed5ea8fc417f8247e689e804e6f1) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:38.821626 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0374a5e4-970c-457e-9165-66a1381c9cf9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:38.823858 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0374a5e4-970c-457e-9165-66a1381c9cf9 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:38.830644 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 229/435] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Jul 6 16:54:38 2022] GET /identity/v3/role_assignments?group.id=1690f24ae1274d30b06a3c3e5c02cf6f&scope.project.id=49b1ed5ea8fc417f8247e689e804e6f1&role.id=7fd63d2d99254c00bb3501173643ee30 => generated 611 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:38.855327 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33033b3-bc4b-4577-9b29-5dc6bd90ba84 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:38.897195 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f2e5da-14ee-4392-a005-7949bd24b701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:38.897859 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2e5da-14ee-4392-a005-7949bd24b701 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:38.898142 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2e5da-14ee-4392-a005-7949bd24b701 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:38.898417 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f2e5da-14ee-4392-a005-7949bd24b701 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:38.974921 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f2e5da-14ee-4392-a005-7949bd24b701 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:38.976426 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f2e5da-14ee-4392-a005-7949bd24b701 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:39.027442 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f487d38-2fbb-4f4e-8ab6-37453198e282 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:39.027785 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f487d38-2fbb-4f4e-8ab6-37453198e282 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:39.028028 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f487d38-2fbb-4f4e-8ab6-37453198e282 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:39.029443 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 230/436] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 16:54:39.036353 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e7ea7-036e-468c-9b28-2ac70efbdc2c None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:39.036641 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e7ea7-036e-468c-9b28-2ac70efbdc2c None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:39.036918 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530e7ea7-036e-468c-9b28-2ac70efbdc2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:39.048269 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:39.055318 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-530e7ea7-036e-468c-9b28-2ac70efbdc2c None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:39.066251 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-530e7ea7-036e-468c-9b28-2ac70efbdc2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:39.085609 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 207/437] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:54:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 06 16:54:39.087016 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ecb1c-305e-4f43-9dfc-35456c2e526e None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:39.087763 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ecb1c-305e-4f43-9dfc-35456c2e526e None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:39.088042 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900ecb1c-305e-4f43-9dfc-35456c2e526e None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:39.089333 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 208/438] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:39.093474 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900ecb1c-305e-4f43-9dfc-35456c2e526e None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:39.141219 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 231/439] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:39.142512 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31badf0-59c5-4118-8d81-73a2ab8cea20 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:39.142805 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31badf0-59c5-4118-8d81-73a2ab8cea20 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:39.143086 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31badf0-59c5-4118-8d81-73a2ab8cea20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:39.165029 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-e31badf0-59c5-4118-8d81-73a2ab8cea20 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:39.167377 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-e31badf0-59c5-4118-8d81-73a2ab8cea20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:39.220932 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900ecb1c-305e-4f43-9dfc-35456c2e526e None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:39.241733 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:39.247760 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 232/440] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:39.248940 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31badf0-59c5-4118-8d81-73a2ab8cea20 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:39.260945 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520c4010-1f9b-4e4d-a211-6597faa12de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:39.260945 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c4010-1f9b-4e4d-a211-6597faa12de8 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:39.260945 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c4010-1f9b-4e4d-a211-6597faa12de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:39.260945 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520c4010-1f9b-4e4d-a211-6597faa12de8 None admin] PATH_INFO: `/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:39.261250 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520c4010-1f9b-4e4d-a211-6597faa12de8 None admin] RBAC: Authorizing `identity:get_project(project_id=49b1ed5ea8fc417f8247e689e804e6f1)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:39.262657 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520c4010-1f9b-4e4d-a211-6597faa12de8 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:39.264364 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 209/441] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:54:39 2022] GET /identity/v3/projects/49b1ed5ea8fc417f8247e689e804e6f1 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:39.269717 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520c4010-1f9b-4e4d-a211-6597faa12de8 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:39.303501 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:39.348047 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31badf0-59c5-4118-8d81-73a2ab8cea20 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:39.366506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520c4010-1f9b-4e4d-a211-6597faa12de8 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:39.381977 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc22253a-f158-40ca-8d70-d62200b4f60b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:39.382765 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22253a-f158-40ca-8d70-d62200b4f60b None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:39.383099 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22253a-f158-40ca-8d70-d62200b4f60b None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:39.383382 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22253a-f158-40ca-8d70-d62200b4f60b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:39.393728 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc22253a-f158-40ca-8d70-d62200b4f60b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:39.400240 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb45a2d-6202-4ffa-95f7-df698a717f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:39.401085 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb45a2d-6202-4ffa-95f7-df698a717f3d None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:39.401368 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb45a2d-6202-4ffa-95f7-df698a717f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:39.401644 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb45a2d-6202-4ffa-95f7-df698a717f3d None admin] PATH_INFO: `/v3/groups/1690f24ae1274d30b06a3c3e5c02cf6f` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:39.401732 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-dc22253a-f158-40ca-8d70-d62200b4f60b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:54:39.401732 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:39.401732 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:39.401732 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:39.401732 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:39.401732 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:39.401732 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:39.401732 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:39.401732 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:39.401732 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:39.401732 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:39.401732 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:39.401732 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:39.401732 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:39.401732 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:39.401732 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:39.402602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:39.402602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:39.402602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:39.402602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:39.402602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:39.402602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:39.402602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:39.402602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:39.402602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:39.402602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:39.402602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:39.402602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:39.402602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:39.402602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:39.402602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:39.402602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:39.403416 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:39.403416 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:39.403416 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:39.403416 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:39.403416 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:39.403416 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:39.403416 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:39.403416 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:39.403416 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:39.403416 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:39.403416 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:54:39.403416 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:39.404027 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb45a2d-6202-4ffa-95f7-df698a717f3d None admin] RBAC: Authorizing `identity:get_group(group_id=1690f24ae1274d30b06a3c3e5c02cf6f)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:39.404094 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 233/442] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 6 16:54:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:39.404094 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc22253a-f158-40ca-8d70-d62200b4f60b None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:39.404275 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb45a2d-6202-4ffa-95f7-df698a717f3d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:39.407766 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 210/443] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 6 16:54:39 2022] GET /identity/v3/groups/1690f24ae1274d30b06a3c3e5c02cf6f => generated 226 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:39.410017 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb45a2d-6202-4ffa-95f7-df698a717f3d None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:39.461167 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:39.505480 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc22253a-f158-40ca-8d70-d62200b4f60b None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:39.530515 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb45a2d-6202-4ffa-95f7-df698a717f3d None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:39.532782 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:39.539871 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb7c5e-71b4-4db4-8b32-71610fdf4175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:39.541290 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb7c5e-71b4-4db4-8b32-71610fdf4175 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:39.541575 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb7c5e-71b4-4db4-8b32-71610fdf4175 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:39.544302 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bb7c5e-71b4-4db4-8b32-71610fdf4175 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:39.551983 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bb7c5e-71b4-4db4-8b32-71610fdf4175 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:39.560048 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-b9bb7c5e-71b4-4db4-8b32-71610fdf4175 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 16:54:39.560048 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:39.560048 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:39.560048 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:39.560048 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:39.560048 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:39.560048 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:39.560048 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:39.560048 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:39.560048 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:39.560048 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:39.560048 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:39.560048 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:39.560048 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:39.560048 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:39.560048 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:39.560957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:39.560957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:39.560957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:39.560957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:39.560957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:39.560957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:39.560957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:39.560957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:39.560957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:39.560957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:39.560957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:39.560957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:39.560957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:39.560957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:39.560957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:39.560957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:39.561850 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:39.561850 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:39.561850 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:39.561850 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:39.561850 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:39.561850 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:39.561850 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:39.561850 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:39.561850 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:39.561850 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:39.561850 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 16:54:39.561850 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:39.563866 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 234/444] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 6 16:54:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:39.564010 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d504426c-bdeb-45a2-ba81-3251daa54970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:39.564559 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504426c-bdeb-45a2-ba81-3251daa54970 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:39.564849 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504426c-bdeb-45a2-ba81-3251daa54970 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:39.564956 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb7c5e-71b4-4db4-8b32-71610fdf4175 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:39.565126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d504426c-bdeb-45a2-ba81-3251daa54970 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:39.565874 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d504426c-bdeb-45a2-ba81-3251daa54970 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:39.566319 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d504426c-bdeb-45a2-ba81-3251daa54970 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:39.567574 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d504426c-bdeb-45a2-ba81-3251daa54970 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:39.575625 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 211/445] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:54:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:39.576479 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d504426c-bdeb-45a2-ba81-3251daa54970 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:39.668286 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bb7c5e-71b4-4db4-8b32-71610fdf4175 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:39.684122 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d504426c-bdeb-45a2-ba81-3251daa54970 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:39.685494 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:39.702185 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a978d7-a6fc-43be-bc02-5cf8241decea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:39.702843 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a978d7-a6fc-43be-bc02-5cf8241decea None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:39.703126 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a978d7-a6fc-43be-bc02-5cf8241decea None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:39.704725 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a978d7-a6fc-43be-bc02-5cf8241decea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:39.704725 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a978d7-a6fc-43be-bc02-5cf8241decea None admin] RBAC: Adding query filter params (group.id=1690f24ae1274d30b06a3c3e5c02cf6f, role.id=c31c5ee7ed8646e48fa81e85d10fcaf2, scope.project.id=49b1ed5ea8fc417f8247e689e804e6f1) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:39.704725 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a978d7-a6fc-43be-bc02-5cf8241decea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:39.706077 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a978d7-a6fc-43be-bc02-5cf8241decea None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:39.711691 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 235/446] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Jul 6 16:54:39 2022] GET /identity/v3/role_assignments?group.id=1690f24ae1274d30b06a3c3e5c02cf6f&scope.project.id=49b1ed5ea8fc417f8247e689e804e6f1&role.id=c31c5ee7ed8646e48fa81e85d10fcaf2 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:39.712993 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a978d7-a6fc-43be-bc02-5cf8241decea None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:39.722079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7cb918-7d72-49c7-9950-6182d8078946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:39.722792 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7cb918-7d72-49c7-9950-6182d8078946 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:39.723081 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7cb918-7d72-49c7-9950-6182d8078946 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:39.723361 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7cb918-7d72-49c7-9950-6182d8078946 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:39.724188 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7cb918-7d72-49c7-9950-6182d8078946 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:39.724502 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7cb918-7d72-49c7-9950-6182d8078946 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:39.725964 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7cb918-7d72-49c7-9950-6182d8078946 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:39.736623 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 212/447] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:54:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:39.742652 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7cb918-7d72-49c7-9950-6182d8078946 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:39.773875 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:39.836303 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a978d7-a6fc-43be-bc02-5cf8241decea None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:39.858674 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7cb918-7d72-49c7-9950-6182d8078946 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:39.876769 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a9190-5461-46bf-9848-02b93ce81961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:39.877670 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a9190-5461-46bf-9848-02b93ce81961 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:39.877777 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a9190-5461-46bf-9848-02b93ce81961 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:39.877972 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a9190-5461-46bf-9848-02b93ce81961 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:39.907365 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5441648-8af4-452a-b956-fa67865e7892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:39.908554 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5441648-8af4-452a-b956-fa67865e7892 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:39.908849 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5441648-8af4-452a-b956-fa67865e7892 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:39.909133 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5441648-8af4-452a-b956-fa67865e7892 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:39.935597 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:39.956161 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45a9190-5461-46bf-9848-02b93ce81961 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:39.957637 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45a9190-5461-46bf-9848-02b93ce81961 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:39.993906 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5441648-8af4-452a-b956-fa67865e7892 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:39.995389 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5441648-8af4-452a-b956-fa67865e7892 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:40.028317 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:40.073858 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 236/448] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:54:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 16:54:40.079278 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a9190-5461-46bf-9848-02b93ce81961 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:40.121258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 213/449] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:54:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 16:54:40.127113 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5441648-8af4-452a-b956-fa67865e7892 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:40.175623 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:40.198774 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a9190-5461-46bf-9848-02b93ce81961 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:40.236705 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b3ff8-bdbe-4ed0-a2a2-97e52ab2b969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:40.237574 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b3ff8-bdbe-4ed0-a2a2-97e52ab2b969 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:40.237857 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b3ff8-bdbe-4ed0-a2a2-97e52ab2b969 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:40.238128 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b3ff8-bdbe-4ed0-a2a2-97e52ab2b969 None admin] PATH_INFO: `/v3/projects/18f61d57ca574df5935e41a1653e029c` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:40.239315 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b3ff8-bdbe-4ed0-a2a2-97e52ab2b969 None admin] RBAC: Authorizing `identity:get_project(project_id=18f61d57ca574df5935e41a1653e029c)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:40.240824 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b3ff8-bdbe-4ed0-a2a2-97e52ab2b969 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:40.242719 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 237/450] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:54:40 2022] GET /identity/v3/projects/18f61d57ca574df5935e41a1653e029c => generated 344 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:40.246936 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5441648-8af4-452a-b956-fa67865e7892 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:40.248415 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b3ff8-bdbe-4ed0-a2a2-97e52ab2b969 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:40.272683 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:40.280800 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88763b1-c7ea-4513-8109-486977b0e7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:40.281651 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88763b1-c7ea-4513-8109-486977b0e7bb None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:40.281933 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88763b1-c7ea-4513-8109-486977b0e7bb None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:40.282208 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88763b1-c7ea-4513-8109-486977b0e7bb None admin] PATH_INFO: `/v3/projects/aa8007020ec543c1a6405283ddd21067` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:40.284474 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88763b1-c7ea-4513-8109-486977b0e7bb None admin] RBAC: Authorizing `identity:get_project(project_id=aa8007020ec543c1a6405283ddd21067)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:40.285928 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88763b1-c7ea-4513-8109-486977b0e7bb None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:40.287638 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 214/451] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:54:40 2022] GET /identity/v3/projects/aa8007020ec543c1a6405283ddd21067 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:40.293637 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88763b1-c7ea-4513-8109-486977b0e7bb None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:40.346311 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b3ff8-bdbe-4ed0-a2a2-97e52ab2b969 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:40.379324 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48af21c-ee47-4262-919d-2bf9cacf7edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:40.380146 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48af21c-ee47-4262-919d-2bf9cacf7edb None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:40.380432 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48af21c-ee47-4262-919d-2bf9cacf7edb None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:40.380707 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48af21c-ee47-4262-919d-2bf9cacf7edb None admin] PATH_INFO: `/v3/groups/cdac86b5299b4addb0e7207f9b5ac342` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:40.381881 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48af21c-ee47-4262-919d-2bf9cacf7edb None admin] RBAC: Authorizing `identity:get_group(group_id=cdac86b5299b4addb0e7207f9b5ac342)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:40.383340 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48af21c-ee47-4262-919d-2bf9cacf7edb None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:40.384994 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 238/452] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 6 16:54:40 2022] GET /identity/v3/groups/cdac86b5299b4addb0e7207f9b5ac342 => generated 229 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:40.387967 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88763b1-c7ea-4513-8109-486977b0e7bb None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:40.390787 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48af21c-ee47-4262-919d-2bf9cacf7edb None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:40.409354 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:40.421776 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56fdda-9410-41f1-a1c1-fadb270e300b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:40.422672 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56fdda-9410-41f1-a1c1-fadb270e300b None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:40.422962 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56fdda-9410-41f1-a1c1-fadb270e300b None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:40.423238 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b56fdda-9410-41f1-a1c1-fadb270e300b None admin] PATH_INFO: `/v3/groups/1690f24ae1274d30b06a3c3e5c02cf6f` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:40.424428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56fdda-9410-41f1-a1c1-fadb270e300b None admin] RBAC: Authorizing `identity:get_group(group_id=1690f24ae1274d30b06a3c3e5c02cf6f)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:40.425853 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b56fdda-9410-41f1-a1c1-fadb270e300b None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:40.427465 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 215/453] 10.222.0.40 () {56 vars in 1180 bytes} [Wed Jul 6 16:54:40 2022] GET /identity/v3/groups/1690f24ae1274d30b06a3c3e5c02cf6f => generated 226 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:40.433221 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56fdda-9410-41f1-a1c1-fadb270e300b None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:40.489268 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48af21c-ee47-4262-919d-2bf9cacf7edb None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:40.492890 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:40.521956 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e4f57-c08f-4a56-a080-d6db2fd77008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:40.522612 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e4f57-c08f-4a56-a080-d6db2fd77008 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:40.522890 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e4f57-c08f-4a56-a080-d6db2fd77008 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:40.523163 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e4f57-c08f-4a56-a080-d6db2fd77008 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:40.524020 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e4f57-c08f-4a56-a080-d6db2fd77008 None admin] RBAC: Adding query filter params (group.id=cdac86b5299b4addb0e7207f9b5ac342, role.id=b41f8a7a58e64f749f2099ca45cc04a7, scope.project.id=18f61d57ca574df5935e41a1653e029c) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:40.524330 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e4f57-c08f-4a56-a080-d6db2fd77008 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:40.526277 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e4f57-c08f-4a56-a080-d6db2fd77008 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:40.530136 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b56fdda-9410-41f1-a1c1-fadb270e300b None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:40.531924 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 239/454] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Jul 6 16:54:40 2022] GET /identity/v3/role_assignments?group.id=cdac86b5299b4addb0e7207f9b5ac342&scope.project.id=18f61d57ca574df5935e41a1653e029c&role.id=b41f8a7a58e64f749f2099ca45cc04a7 => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:40.565116 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec61deb9-6598-4406-9c51-366b0872448c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:40.565791 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61deb9-6598-4406-9c51-366b0872448c None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:40.566076 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61deb9-6598-4406-9c51-366b0872448c None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:40.566355 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61deb9-6598-4406-9c51-366b0872448c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:40.567213 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec61deb9-6598-4406-9c51-366b0872448c None admin] RBAC: Adding query filter params (group.id=1690f24ae1274d30b06a3c3e5c02cf6f, role.id=c31c5ee7ed8646e48fa81e85d10fcaf2, scope.project.id=aa8007020ec543c1a6405283ddd21067) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:40.567532 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec61deb9-6598-4406-9c51-366b0872448c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:40.569060 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec61deb9-6598-4406-9c51-366b0872448c None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:40.587808 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 216/455] 10.222.0.40 () {56 vars in 1376 bytes} [Wed Jul 6 16:54:40 2022] GET /identity/v3/role_assignments?group.id=1690f24ae1274d30b06a3c3e5c02cf6f&scope.project.id=aa8007020ec543c1a6405283ddd21067&role.id=c31c5ee7ed8646e48fa81e85d10fcaf2 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:43.333346 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc60a9c-755c-436e-bdc9-162ae9f2e0fb None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:43.334330 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc60a9c-755c-436e-bdc9-162ae9f2e0fb None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:43.334819 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc60a9c-755c-436e-bdc9-162ae9f2e0fb None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:43.336680 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 240/456] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:43.341792 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ecc8b-3661-4781-b6f4-eba08d5ee372 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:43.342610 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ecc8b-3661-4781-b6f4-eba08d5ee372 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:43.343086 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ecc8b-3661-4781-b6f4-eba08d5ee372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:43.364890 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-3a1ecc8b-3661-4781-b6f4-eba08d5ee372 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:43.366457 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-3a1ecc8b-3661-4781-b6f4-eba08d5ee372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:43.402435 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb42515-6078-464b-a4e0-dc8c94499e15 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:43.403050 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb42515-6078-464b-a4e0-dc8c94499e15 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:43.403547 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb42515-6078-464b-a4e0-dc8c94499e15 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:43.406407 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 241/457] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:43.410500 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3afcc0-ca4b-4861-90bd-43c4c86a6e35 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:43.411020 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3afcc0-ca4b-4861-90bd-43c4c86a6e35 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:43.411515 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3afcc0-ca4b-4861-90bd-43c4c86a6e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:43.438330 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-1a3afcc0-ca4b-4861-90bd-43c4c86a6e35 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:43.439510 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-1a3afcc0-ca4b-4861-90bd-43c4c86a6e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:43.476939 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 217/458] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:43.524856 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1ecc8b-3661-4781-b6f4-eba08d5ee372 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:43.542696 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 242/459] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 16:54:43.566894 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3afcc0-ca4b-4861-90bd-43c4c86a6e35 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:43.591199 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:43.624970 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:43.670256 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1ecc8b-3661-4781-b6f4-eba08d5ee372 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:43.696684 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3afcc0-ca4b-4861-90bd-43c4c86a6e35 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:43.713576 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749147d4-796e-4a03-a50d-68002648487c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:43.714626 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749147d4-796e-4a03-a50d-68002648487c None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:43.715086 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749147d4-796e-4a03-a50d-68002648487c None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:43.715510 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749147d4-796e-4a03-a50d-68002648487c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:43.729523 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749147d4-796e-4a03-a50d-68002648487c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:43.731483 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749147d4-796e-4a03-a50d-68002648487c None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:43.753209 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-749147d4-796e-4a03-a50d-68002648487c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:43.753209 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:43.753209 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:43.753209 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:43.753209 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:43.753209 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:43.753209 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:43.753209 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:43.753209 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:43.753209 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:43.753209 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:43.753209 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:43.753209 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:54:43.753209 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:54:43.753209 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:54:43.754290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:54:43.754290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:43.754290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:43.754290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:43.754290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:43.754290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:43.754290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:43.754290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:43.754290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:43.754290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:43.754290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:43.754290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:43.754290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:43.754290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:43.754290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:43.754290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:43.754290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:43.755161 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:54:43.755161 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:54:43.755161 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:43.755161 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:43.756055 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1959a5-2d1a-40f4-bc55-a4b0d3ae66dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:43.756856 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1959a5-2d1a-40f4-bc55-a4b0d3ae66dc None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:43.757143 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1959a5-2d1a-40f4-bc55-a4b0d3ae66dc None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:43.757423 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1959a5-2d1a-40f4-bc55-a4b0d3ae66dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:43.758268 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 218/460] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:54:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:43.759918 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd9d8a1-f359-46f1-b612-64717fe1d9c0 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:43.760474 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd9d8a1-f359-46f1-b612-64717fe1d9c0 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:43.760969 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd9d8a1-f359-46f1-b612-64717fe1d9c0 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:43.762568 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 219/461] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:43.768207 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd9d8a1-f359-46f1-b612-64717fe1d9c0 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:43.773768 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1959a5-2d1a-40f4-bc55-a4b0d3ae66dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:43.775290 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1959a5-2d1a-40f4-bc55-a4b0d3ae66dc None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:43.794389 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-9b1959a5-2d1a-40f4-bc55-a4b0d3ae66dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:43.794389 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:43.794389 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:43.794389 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:43.794389 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:43.794389 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:43.794389 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:43.794389 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:43.794389 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:43.794389 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:43.794389 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:43.794389 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:43.794389 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:54:43.794389 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:54:43.794389 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:54:43.795320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:54:43.795320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:43.795320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:43.795320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:43.795320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:43.795320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:43.795320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:43.795320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:43.795320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:43.795320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:43.795320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:43.795320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:43.795320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:43.795320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:43.795320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:43.795320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:43.795320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:43.796203 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:54:43.796203 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:54:43.796203 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:43.796203 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:43.796203 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 243/462] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:54:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:54:43.797001 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97f41c-6394-40b0-9564-ed11f7354fc6 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:43.797233 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97f41c-6394-40b0-9564-ed11f7354fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:43.797509 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc97f41c-6394-40b0-9564-ed11f7354fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:43.815110 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-fc97f41c-6394-40b0-9564-ed11f7354fc6 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:43.816341 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-fc97f41c-6394-40b0-9564-ed11f7354fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:43.878954 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:43.886453 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd9d8a1-f359-46f1-b612-64717fe1d9c0 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:43.888271 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 244/463] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:43.893751 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97f41c-6394-40b0-9564-ed11f7354fc6 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:43.910922 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:43.960733 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93921e3-a435-4ed7-9faa-4d424f7dbe47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:43.961406 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93921e3-a435-4ed7-9faa-4d424f7dbe47 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:43.961694 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93921e3-a435-4ed7-9faa-4d424f7dbe47 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:43.961961 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93921e3-a435-4ed7-9faa-4d424f7dbe47 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:43.962727 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93921e3-a435-4ed7-9faa-4d424f7dbe47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:43.963036 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93921e3-a435-4ed7-9faa-4d424f7dbe47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:43.964441 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93921e3-a435-4ed7-9faa-4d424f7dbe47 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:43.977679 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 220/464] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:54:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:43.979066 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93921e3-a435-4ed7-9faa-4d424f7dbe47 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:44.016413 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc97f41c-6394-40b0-9564-ed11f7354fc6 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:44.053532 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc8e98-fe33-4a40-b513-d9030f284275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:44.054212 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc8e98-fe33-4a40-b513-d9030f284275 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:44.054498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc8e98-fe33-4a40-b513-d9030f284275 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:44.054777 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dc8e98-fe33-4a40-b513-d9030f284275 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:44.055505 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dc8e98-fe33-4a40-b513-d9030f284275 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:44.055887 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dc8e98-fe33-4a40-b513-d9030f284275 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:44.057264 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dc8e98-fe33-4a40-b513-d9030f284275 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:44.069449 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 245/465] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:54:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:44.070238 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc8e98-fe33-4a40-b513-d9030f284275 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:44.097793 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93921e3-a435-4ed7-9faa-4d424f7dbe47 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:44.129429 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:44.134694 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20535df-2699-49e6-b5a5-216cc27c67f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:44.135781 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20535df-2699-49e6-b5a5-216cc27c67f2 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:44.136154 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20535df-2699-49e6-b5a5-216cc27c67f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:44.136441 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20535df-2699-49e6-b5a5-216cc27c67f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:44.150715 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20535df-2699-49e6-b5a5-216cc27c67f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:44.152265 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20535df-2699-49e6-b5a5-216cc27c67f2 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:44.171088 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-b20535df-2699-49e6-b5a5-216cc27c67f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:44.171088 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:44.171088 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:44.171088 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:44.171088 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:44.171088 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:44.171088 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:44.171088 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:44.171088 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:44.171088 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:44.171088 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:44.171088 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:44.171088 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:54:44.171088 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:54:44.171088 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:54:44.171999 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:54:44.171999 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:44.171999 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:44.171999 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:44.171999 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:44.171999 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:44.171999 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:44.171999 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:44.171999 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:44.171999 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:44.171999 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:44.171999 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:44.171999 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:44.171999 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:44.171999 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:44.171999 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:44.171999 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:44.172883 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:54:44.172883 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:54:44.172883 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:44.172883 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:44.172883 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 221/466] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:54:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:44.173691 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b10c8-0e70-46a3-9e68-fbaea186c19d None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:44.173895 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b10c8-0e70-46a3-9e68-fbaea186c19d None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:44.174176 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b10c8-0e70-46a3-9e68-fbaea186c19d None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:44.175591 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 222/467] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:44.176141 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b10c8-0e70-46a3-9e68-fbaea186c19d None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:44.182803 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dc8e98-fe33-4a40-b513-d9030f284275 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:44.207765 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:44.222275 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c6c87e-c751-457a-8d13-f835ef6414d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:44.222973 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6c87e-c751-457a-8d13-f835ef6414d2 None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:44.223264 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6c87e-c751-457a-8d13-f835ef6414d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:44.223752 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c6c87e-c751-457a-8d13-f835ef6414d2 None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:44.225703 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c6c87e-c751-457a-8d13-f835ef6414d2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:44.229290 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c6c87e-c751-457a-8d13-f835ef6414d2 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:44.291317 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b10c8-0e70-46a3-9e68-fbaea186c19d None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:44.326304 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a6fcbf-5794-435f-a031-04e1f6b251b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:44.326968 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a6fcbf-5794-435f-a031-04e1f6b251b2 None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:44.327251 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a6fcbf-5794-435f-a031-04e1f6b251b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:44.327526 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a6fcbf-5794-435f-a031-04e1f6b251b2 None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:44.328264 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a6fcbf-5794-435f-a031-04e1f6b251b2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:44.329708 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a6fcbf-5794-435f-a031-04e1f6b251b2 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:44.338047 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 246/468] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 6 16:54:44 2022] POST /identity/v3/users => generated 246 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 16:54:44.339306 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e9da2-8f6c-4f8f-949b-43e64b23e808 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:44.339596 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e9da2-8f6c-4f8f-949b-43e64b23e808 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:44.339903 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e9da2-8f6c-4f8f-949b-43e64b23e808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:44.360832 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-cf8e9da2-8f6c-4f8f-949b-43e64b23e808 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:44.362082 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-cf8e9da2-8f6c-4f8f-949b-43e64b23e808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:44.424254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 223/469] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 6 16:54:44 2022] POST /identity/v3/users => generated 248 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 16:54:44.442766 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 247/470] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:44.443009 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a6fcbf-5794-435f-a031-04e1f6b251b2 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:44.443103 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225208c-e32c-4db2-be07-88649659119f None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:44.443103 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225208c-e32c-4db2-be07-88649659119f None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:44.443103 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f225208c-e32c-4db2-be07-88649659119f None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:44.444008 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 248/471] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:44.445485 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee10685-ebc7-448b-8967-58c8bfc7623a None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:44.445805 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee10685-ebc7-448b-8967-58c8bfc7623a None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:44.446317 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee10685-ebc7-448b-8967-58c8bfc7623a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:44.465144 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-bee10685-ebc7-448b-8967-58c8bfc7623a None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:44.466519 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-bee10685-ebc7-448b-8967-58c8bfc7623a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:44.511523 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:44.560808 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 249/472] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:44.561998 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee10685-ebc7-448b-8967-58c8bfc7623a None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:44.594905 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:44.624021 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a6fcbf-5794-435f-a031-04e1f6b251b2 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:44.668343 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1e48a-1810-4518-af15-d61587fab337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:44.669007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1e48a-1810-4518-af15-d61587fab337 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:44.669291 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1e48a-1810-4518-af15-d61587fab337 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:44.669562 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea1e48a-1810-4518-af15-d61587fab337 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:44.670284 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea1e48a-1810-4518-af15-d61587fab337 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:44.670586 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea1e48a-1810-4518-af15-d61587fab337 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:44.673386 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea1e48a-1810-4518-af15-d61587fab337 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:44.685863 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 224/473] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:54:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:44.686519 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1e48a-1810-4518-af15-d61587fab337 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:44.695074 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee10685-ebc7-448b-8967-58c8bfc7623a None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:44.736486 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf7492b-6283-4db9-8eed-bffd46a12acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:44.737298 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7492b-6283-4db9-8eed-bffd46a12acd None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:44.737583 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7492b-6283-4db9-8eed-bffd46a12acd None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:44.737860 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf7492b-6283-4db9-8eed-bffd46a12acd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:44.751448 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf7492b-6283-4db9-8eed-bffd46a12acd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:44.752950 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf7492b-6283-4db9-8eed-bffd46a12acd None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:44.771635 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-5bf7492b-6283-4db9-8eed-bffd46a12acd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:44.771635 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:44.771635 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:44.771635 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:44.771635 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:44.771635 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:44.771635 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:44.771635 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:44.771635 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:44.771635 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:44.771635 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:44.771635 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:44.771635 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:54:44.771635 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:54:44.771635 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:54:44.772707 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:54:44.772707 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:44.772707 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:44.772707 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:44.772707 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:44.772707 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:44.772707 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:44.772707 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:44.772707 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:44.772707 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:44.772707 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:44.772707 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:44.772707 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:44.772707 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:44.772707 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:44.772707 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:44.772707 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:44.773535 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:54:44.773535 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:54:44.773535 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:44.773535 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:44.773535 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 250/474] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:54:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:44.773791 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf7492b-6283-4db9-8eed-bffd46a12acd None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:44.800321 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:44.824938 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea1e48a-1810-4518-af15-d61587fab337 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:44.871759 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6547d5-9e46-4b4e-b213-916a41493b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:44.873467 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6547d5-9e46-4b4e-b213-916a41493b35 None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:44.873961 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6547d5-9e46-4b4e-b213-916a41493b35 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:44.874639 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6547d5-9e46-4b4e-b213-916a41493b35 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:44.876276 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6547d5-9e46-4b4e-b213-916a41493b35 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:44.877873 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf7492b-6283-4db9-8eed-bffd46a12acd None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:44.877989 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6547d5-9e46-4b4e-b213-916a41493b35 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:44.914938 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266c63c-4839-4df8-a1e0-65a9c93d12d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:44.915948 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266c63c-4839-4df8-a1e0-65a9c93d12d7 None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:44.916587 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266c63c-4839-4df8-a1e0-65a9c93d12d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:44.917014 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d266c63c-4839-4df8-a1e0-65a9c93d12d7 None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:44.917870 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d266c63c-4839-4df8-a1e0-65a9c93d12d7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:44.919538 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d266c63c-4839-4df8-a1e0-65a9c93d12d7 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:44.932309 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.sql.core [None req-7d6547d5-9e46-4b4e-b213-916a41493b35 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 06 16:54:44.932309 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 06 16:54:44.932309 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [parameters: {'id': '25ceca4f0c0d4ddaa55c8f50c04af91e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 06 16:54:44.932309 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80470) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 06 16:54:44.950482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-7d6547d5-9e46-4b4e-b213-916a41493b35 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 06 16:54:44.950482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:44.950482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 06 16:54:44.950482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 06 16:54:44.950482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 06 16:54:44.950482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 06 16:54:44.950482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 06 16:54:44.950482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application result = self._query(query) Jul 06 16:54:44.950482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 06 16:54:44.950482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application conn.query(q) Jul 06 16:54:44.950482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 06 16:54:44.950482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 06 16:54:44.950482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 06 16:54:44.950482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application result.read() Jul 06 16:54:44.950482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 06 16:54:44.951455 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 06 16:54:44.951455 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 06 16:54:44.951455 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 06 16:54:44.951455 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 06 16:54:44.951455 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 06 16:54:44.951455 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 06 16:54:44.951455 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 06 16:54:44.951455 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 06 16:54:44.951455 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:44.951455 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 06 16:54:44.951455 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:44.951455 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:44.951455 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 06 16:54:44.951455 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 06 16:54:44.951455 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 06 16:54:44.951455 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return ref.to_dict() Jul 06 16:54:44.951455 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 06 16:54:44.951455 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application next(self.gen) Jul 06 16:54:44.953027 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 06 16:54:44.953027 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application yield resource Jul 06 16:54:44.953027 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 06 16:54:44.953027 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application next(self.gen) Jul 06 16:54:44.953027 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 06 16:54:44.953027 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self.session.rollback() Jul 06 16:54:44.953027 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 06 16:54:44.953027 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self.force_reraise() Jul 06 16:54:44.953027 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 06 16:54:44.953027 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise self.value Jul 06 16:54:44.953027 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 06 16:54:44.953027 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 06 16:54:44.953027 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 06 16:54:44.953027 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application session.commit() Jul 06 16:54:44.953027 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 06 16:54:44.953027 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 06 16:54:44.953027 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 06 16:54:44.953912 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._prepare_impl() Jul 06 16:54:44.953912 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 06 16:54:44.953912 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self.session.flush() Jul 06 16:54:44.953912 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 06 16:54:44.953912 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._flush(objects) Jul 06 16:54:44.953912 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 06 16:54:44.953912 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 06 16:54:44.953912 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 06 16:54:44.953912 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application compat.raise_( Jul 06 16:54:44.953912 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 06 16:54:44.953912 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception Jul 06 16:54:44.953912 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 06 16:54:44.953912 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application flush_context.execute() Jul 06 16:54:44.953912 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 06 16:54:44.953912 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rec.execute(self) Jul 06 16:54:44.953912 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 06 16:54:44.953912 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 06 16:54:44.954793 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 06 16:54:44.954793 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 06 16:54:44.954793 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 06 16:54:44.954793 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 06 16:54:44.954793 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 06 16:54:44.954793 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 06 16:54:44.954793 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 06 16:54:44.954793 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 06 16:54:44.954793 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 06 16:54:44.954793 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 06 16:54:44.954793 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 06 16:54:44.954793 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 06 16:54:44.954793 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 06 16:54:44.954793 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 06 16:54:44.954793 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 06 16:54:44.954793 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception Jul 06 16:54:44.957318 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 06 16:54:44.957318 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 06 16:54:44.957318 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 06 16:54:44.957318 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 06 16:54:44.957318 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 06 16:54:44.957318 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application result = self._query(query) Jul 06 16:54:44.957318 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 06 16:54:44.957318 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application conn.query(q) Jul 06 16:54:44.957318 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 06 16:54:44.957318 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 06 16:54:44.957318 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 06 16:54:44.957318 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application result.read() Jul 06 16:54:44.957318 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 06 16:54:44.957318 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 06 16:54:44.957318 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 06 16:54:44.957318 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 06 16:54:44.959285 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 06 16:54:44.959285 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 06 16:54:44.959285 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 06 16:54:44.959285 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 06 16:54:44.959285 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 06 16:54:44.959285 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 06 16:54:44.959285 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [parameters: {'id': '25ceca4f0c0d4ddaa55c8f50c04af91e', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 06 16:54:44.959285 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 06 16:54:44.959285 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:44.959285 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 16:54:44.959285 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:44.959285 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:44.959285 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:44.959285 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:44.959285 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:44.959285 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:44.962680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:44.962680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:44.962680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:44.962680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:44.962680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:44.962680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:44.962680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 06 16:54:44.962680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 06 16:54:44.962680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:44.962680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:44.962680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 06 16:54:44.962680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 06 16:54:44.962680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 06 16:54:44.962680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 06 16:54:44.962680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 06 16:54:44.962680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:44.964720 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 225/475] 10.222.0.40 () {60 vars in 1131 bytes} [Wed Jul 6 16:54:44 2022] POST /identity/v3/roles => generated 144 bytes in 266 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:44.964720 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6547d5-9e46-4b4e-b213-916a41493b35 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:44.964720 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:45.041377 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 251/476] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 6 16:54:44 2022] POST /identity/v3/users => generated 248 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 16:54:45.042646 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266c63c-4839-4df8-a1e0-65a9c93d12d7 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:45.092444 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6547d5-9e46-4b4e-b213-916a41493b35 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:45.185989 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:45.203617 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2549401b-7433-401f-9256-242072b46bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:45.203617 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549401b-7433-401f-9256-242072b46bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:45.204907 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549401b-7433-401f-9256-242072b46bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:45.205455 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2549401b-7433-401f-9256-242072b46bf6 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:45.207358 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2549401b-7433-401f-9256-242072b46bf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:45.207804 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2549401b-7433-401f-9256-242072b46bf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:45.209616 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2549401b-7433-401f-9256-242072b46bf6 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:45.217175 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d266c63c-4839-4df8-a1e0-65a9c93d12d7 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:45.234657 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 226/477] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:54:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:45.240587 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2549401b-7433-401f-9256-242072b46bf6 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:45.273635 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc129f-4e12-4505-8d9b-c28c9c3dd180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:45.274629 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc129f-4e12-4505-8d9b-c28c9c3dd180 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:45.275035 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc129f-4e12-4505-8d9b-c28c9c3dd180 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:45.275407 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc129f-4e12-4505-8d9b-c28c9c3dd180 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:45.286154 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dc129f-4e12-4505-8d9b-c28c9c3dd180 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:45.295685 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-07dc129f-4e12-4505-8d9b-c28c9c3dd180 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 16:54:45.295685 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:45.295685 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:45.295685 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:45.295685 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:45.295685 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:45.295685 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:45.295685 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:45.295685 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:45.295685 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:45.295685 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:45.295685 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:45.295685 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:45.295685 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:45.295685 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:45.295685 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:45.296859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:45.296859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:45.296859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:45.296859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:45.296859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:45.296859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:45.296859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:45.296859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:45.296859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:45.296859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:45.296859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:45.296859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:45.296859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:45.296859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:45.296859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:45.296859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:45.298178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:45.298178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:45.298178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:45.298178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:45.298178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:45.298178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:45.298178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:45.298178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:45.298178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:45.298178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:45.298178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 06 16:54:45.298178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:45.299330 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 252/478] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 6 16:54:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:45.304899 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc129f-4e12-4505-8d9b-c28c9c3dd180 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:45.372516 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:45.419150 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2549401b-7433-401f-9256-242072b46bf6 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:45.437139 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc129f-4e12-4505-8d9b-c28c9c3dd180 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:45.456973 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1b28f4-3562-4a5e-a3da-976eb371c2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:45.457624 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b28f4-3562-4a5e-a3da-976eb371c2c6 None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:45.457906 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b28f4-3562-4a5e-a3da-976eb371c2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:45.458197 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b28f4-3562-4a5e-a3da-976eb371c2c6 None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:45.458932 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1b28f4-3562-4a5e-a3da-976eb371c2c6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:45.460435 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1b28f4-3562-4a5e-a3da-976eb371c2c6 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:45.487492 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94344f50-d5cb-4642-a386-2e657bf71fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:45.488582 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94344f50-d5cb-4642-a386-2e657bf71fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:45.489121 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94344f50-d5cb-4642-a386-2e657bf71fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:45.489603 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94344f50-d5cb-4642-a386-2e657bf71fe0 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:45.490673 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94344f50-d5cb-4642-a386-2e657bf71fe0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:45.491237 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94344f50-d5cb-4642-a386-2e657bf71fe0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:45.492878 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94344f50-d5cb-4642-a386-2e657bf71fe0 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:45.499759 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:45.505020 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 253/479] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:54:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:45.542214 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 227/480] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 6 16:54:45 2022] POST /identity/v3/users => generated 249 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 16:54:46.621771 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86535df-c42c-4290-b68e-98962b4ad284 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:46.622290 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86535df-c42c-4290-b68e-98962b4ad284 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:46.622290 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86535df-c42c-4290-b68e-98962b4ad284 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:46.623937 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 254/481] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:46.629176 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bfde6d-61d9-4bdf-aae0-2a5c16d07224 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:46.629596 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bfde6d-61d9-4bdf-aae0-2a5c16d07224 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:46.629741 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bfde6d-61d9-4bdf-aae0-2a5c16d07224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:46.656718 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-48bfde6d-61d9-4bdf-aae0-2a5c16d07224 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:46.657900 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-48bfde6d-61d9-4bdf-aae0-2a5c16d07224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:46.753263 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 228/482] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 16:54:46.788647 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86535df-c42c-4290-b68e-98962b4ad284 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:46.937533 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86535df-c42c-4290-b68e-98962b4ad284 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:46.963117 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0254c-dcfe-46fd-ae73-efc5965f982e None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:46.963428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0254c-dcfe-46fd-ae73-efc5965f982e None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:46.963694 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac0254c-dcfe-46fd-ae73-efc5965f982e None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:46.965283 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 229/483] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:46.980765 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7c72c-ce04-422e-9706-569e9683b203 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:46.981051 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7c72c-ce04-422e-9706-569e9683b203 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:46.981331 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7c72c-ce04-422e-9706-569e9683b203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:46.982767 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa10c72-e19c-4c97-8051-58f0fb0ee029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:46.983587 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa10c72-e19c-4c97-8051-58f0fb0ee029 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:46.983907 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa10c72-e19c-4c97-8051-58f0fb0ee029 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:46.984192 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa10c72-e19c-4c97-8051-58f0fb0ee029 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:46.995824 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa10c72-e19c-4c97-8051-58f0fb0ee029 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:47.003787 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-1aa10c72-e19c-4c97-8051-58f0fb0ee029 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 16:54:47.003787 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:47.003787 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:47.003787 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:47.003787 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:47.003787 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:47.003787 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:47.003787 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:47.003787 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:47.003787 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:47.003787 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:47.003787 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:47.003787 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:47.003787 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:47.003787 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:47.003787 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:47.004661 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:47.004661 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:47.004661 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:47.004661 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:47.004661 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:47.004661 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:47.004661 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:47.004661 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:47.004661 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:47.004661 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:47.004661 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:47.004661 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:47.004661 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:47.004661 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:47.004661 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:47.004661 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:47.005466 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:47.005466 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:47.005466 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:47.005466 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:47.005466 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:47.005466 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:47.005466 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:47.005466 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:47.005466 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:47.005466 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:47.005466 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 16:54:47.005466 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:47.005466 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 255/484] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 6 16:54:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:47.012276 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa10c72-e19c-4c97-8051-58f0fb0ee029 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:47.013544 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-24c7c72c-ce04-422e-9706-569e9683b203 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:47.019953 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-24c7c72c-ce04-422e-9706-569e9683b203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:47.102146 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:47.138865 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa10c72-e19c-4c97-8051-58f0fb0ee029 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:47.144067 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:47.182496 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434d5e7-ac75-4439-91ef-40516c4eaa67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:47.183185 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434d5e7-ac75-4439-91ef-40516c4eaa67 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:47.183477 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434d5e7-ac75-4439-91ef-40516c4eaa67 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:47.183780 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434d5e7-ac75-4439-91ef-40516c4eaa67 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:47.186944 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6434d5e7-ac75-4439-91ef-40516c4eaa67 None admin] RBAC: Adding query filter params (name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:47.187261 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6434d5e7-ac75-4439-91ef-40516c4eaa67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:47.188742 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6434d5e7-ac75-4439-91ef-40516c4eaa67 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:47.204944 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 230/485] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:47.207172 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 256/486] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 6 16:54:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:47.213149 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c7c72c-ce04-422e-9706-569e9683b203 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:47.230847 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434d5e7-ac75-4439-91ef-40516c4eaa67 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:47.366294 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434d5e7-ac75-4439-91ef-40516c4eaa67 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:47.420087 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:47.429142 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c7c72c-ce04-422e-9706-569e9683b203 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:47.437071 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a0392-a23c-4578-a8c7-9230fe91eba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:47.437882 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a0392-a23c-4578-a8c7-9230fe91eba2 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:47.438168 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a0392-a23c-4578-a8c7-9230fe91eba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:47.438449 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a0392-a23c-4578-a8c7-9230fe91eba2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:47.462769 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305a0392-a23c-4578-a8c7-9230fe91eba2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:47.473772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-305a0392-a23c-4578-a8c7-9230fe91eba2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 16:54:47.473772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:47.473772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:47.473772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:47.473772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:47.473772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:47.473772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:47.473772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:47.473772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:47.473772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:47.473772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:47.473772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:47.473772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:47.473772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:47.473772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:47.473772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:47.474690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:47.474690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:47.474690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:47.474690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:47.474690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:47.474690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:47.474690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:47.474690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:47.474690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:47.474690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:47.474690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:47.474690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:47.474690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:47.474690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:47.474690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:47.474690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:47.475498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:47.475498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:47.475498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:47.475498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:47.475498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:47.475498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:47.475498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:47.475498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:47.475498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:47.475498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:47.475498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 16:54:47.475498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:47.476136 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 257/487] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 6 16:54:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:54:47.483545 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a0392-a23c-4578-a8c7-9230fe91eba2 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:47.510167 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3084e72-692a-41de-b372-b2ac79984e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:47.511308 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3084e72-692a-41de-b372-b2ac79984e04 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:47.511861 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3084e72-692a-41de-b372-b2ac79984e04 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:47.512370 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3084e72-692a-41de-b372-b2ac79984e04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:47.531143 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3084e72-692a-41de-b372-b2ac79984e04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:47.532954 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3084e72-692a-41de-b372-b2ac79984e04 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:47.551829 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-e3084e72-692a-41de-b372-b2ac79984e04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:47.551829 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:47.551829 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:47.551829 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:47.551829 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:47.551829 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:47.551829 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:47.551829 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:47.551829 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:47.551829 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:47.551829 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:47.551829 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:47.551829 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:54:47.551829 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:54:47.551829 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:54:47.552703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:54:47.552703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:47.552703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:47.552703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:47.552703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:47.552703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:47.552703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:47.552703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:47.552703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:47.552703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:47.552703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:47.552703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:47.552703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:47.552703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:47.552703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:47.552703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:47.552703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:47.553547 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:54:47.553547 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:54:47.553547 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:47.553547 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:47.553547 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 231/488] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:54:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:47.555220 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b9bee-2847-456e-a464-1d699a747d83 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:47.555589 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b9bee-2847-456e-a464-1d699a747d83 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:47.556004 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b9bee-2847-456e-a464-1d699a747d83 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:47.563401 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 232/489] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:47.565039 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b9bee-2847-456e-a464-1d699a747d83 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:47.604065 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:47.622689 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a0392-a23c-4578-a8c7-9230fe91eba2 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:47.670091 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca142cc5-dd47-4ddd-a865-7f4888913eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:47.671103 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca142cc5-dd47-4ddd-a865-7f4888913eaf None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:47.671585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca142cc5-dd47-4ddd-a865-7f4888913eaf None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:47.672086 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca142cc5-dd47-4ddd-a865-7f4888913eaf None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:47.673013 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca142cc5-dd47-4ddd-a865-7f4888913eaf None admin] RBAC: Adding query filter params (name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:47.673503 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca142cc5-dd47-4ddd-a865-7f4888913eaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:47.675065 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca142cc5-dd47-4ddd-a865-7f4888913eaf None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:47.688096 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 258/490] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 6 16:54:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:47.689655 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a83bad-5282-4ad9-b58e-6f4a68f8b796 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:47.690161 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a83bad-5282-4ad9-b58e-6f4a68f8b796 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:47.690621 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a83bad-5282-4ad9-b58e-6f4a68f8b796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:47.698291 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b9bee-2847-456e-a464-1d699a747d83 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:47.708586 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-49a83bad-5282-4ad9-b58e-6f4a68f8b796 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:47.710035 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-49a83bad-5282-4ad9-b58e-6f4a68f8b796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:47.747442 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d24faa0-4610-4c7c-b164-5bd318a77e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:47.748290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24faa0-4610-4c7c-b164-5bd318a77e33 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:47.748548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24faa0-4610-4c7c-b164-5bd318a77e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:47.748823 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24faa0-4610-4c7c-b164-5bd318a77e33 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:47.749545 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d24faa0-4610-4c7c-b164-5bd318a77e33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:47.749849 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d24faa0-4610-4c7c-b164-5bd318a77e33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:47.751421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d24faa0-4610-4c7c-b164-5bd318a77e33 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:47.757591 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:47.769358 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 233/491] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:54:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:47.770284 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d24faa0-4610-4c7c-b164-5bd318a77e33 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:47.786615 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 259/492] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:47.787789 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a83bad-5282-4ad9-b58e-6f4a68f8b796 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:47.871518 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:47.885418 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d24faa0-4610-4c7c-b164-5bd318a77e33 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:47.890776 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a83bad-5282-4ad9-b58e-6f4a68f8b796 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:47.927561 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efce17d-ab10-48bf-b42a-c9207eedbd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:47.928145 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea162d1-e2b9-4467-8068-7661edba47f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:47.928399 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efce17d-ab10-48bf-b42a-c9207eedbd56 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:47.928684 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efce17d-ab10-48bf-b42a-c9207eedbd56 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:47.928958 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efce17d-ab10-48bf-b42a-c9207eedbd56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:47.929807 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea162d1-e2b9-4467-8068-7661edba47f4 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:47.930294 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea162d1-e2b9-4467-8068-7661edba47f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:47.931411 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea162d1-e2b9-4467-8068-7661edba47f4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:47.939615 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea162d1-e2b9-4467-8068-7661edba47f4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:47.941149 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efce17d-ab10-48bf-b42a-c9207eedbd56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:47.941587 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea162d1-e2b9-4467-8068-7661edba47f4 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:47.942586 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efce17d-ab10-48bf-b42a-c9207eedbd56 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:47.959623 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-aea162d1-e2b9-4467-8068-7661edba47f4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 06 16:54:47.959623 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:47.959623 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:47.959623 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:47.959623 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:47.959623 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:47.959623 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:47.959623 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:47.959623 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:47.959623 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:47.959623 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:47.959623 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:47.959623 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 16:54:47.959623 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 16:54:47.959623 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 16:54:47.959623 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 16:54:47.960640 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:47.960640 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:47.960640 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 16:54:47.960640 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:54:47.960640 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 16:54:47.960640 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:54:47.960640 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:47.960640 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:47.960640 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:47.960640 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:47.960640 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:47.960640 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:47.960640 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:47.960640 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:47.960640 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:47.960640 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:47.960640 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:47.960640 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:47.961516 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:47.961516 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:47.961516 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 16:54:47.961516 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 16:54:47.961516 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 16:54:47.961516 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 16:54:47.961516 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 16:54:47.961516 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 16:54:47.961516 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 06 16:54:47.961516 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:47.962988 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-9efce17d-ab10-48bf-b42a-c9207eedbd56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:47.962988 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:47.962988 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:47.962988 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:47.962988 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:47.962988 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:47.962988 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:47.962988 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:47.962988 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:47.962988 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:47.962988 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:47.962988 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:47.962988 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:54:47.962988 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:54:47.962988 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:54:47.963799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:54:47.963799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:47.963799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:47.963799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:47.963799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:47.963799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:47.963799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:47.963799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:47.963799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:47.963799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:47.963799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:47.963799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:47.963799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:47.963799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:47.963799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:47.963799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:47.963799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:47.964637 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:54:47.964637 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:54:47.964637 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:47.964637 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:47.966109 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 260/494] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Jul 6 16:54:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:47.967120 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea162d1-e2b9-4467-8068-7661edba47f4 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:47.967797 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 234/494] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:54:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:47.971578 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efce17d-ab10-48bf-b42a-c9207eedbd56 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:48.009534 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:48.086006 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea162d1-e2b9-4467-8068-7661edba47f4 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:48.104214 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efce17d-ab10-48bf-b42a-c9207eedbd56 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:48.121218 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c82e987-c61c-4870-9c94-4fc0c173399a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:48.122019 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82e987-c61c-4870-9c94-4fc0c173399a None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:48.122302 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82e987-c61c-4870-9c94-4fc0c173399a None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:48.122575 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82e987-c61c-4870-9c94-4fc0c173399a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:48.132886 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c82e987-c61c-4870-9c94-4fc0c173399a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:48.139506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba030e1-2b93-4689-bc88-428154da8a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:48.140210 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba030e1-2b93-4689-bc88-428154da8a2e None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:48.140497 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba030e1-2b93-4689-bc88-428154da8a2e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:48.140776 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba030e1-2b93-4689-bc88-428154da8a2e None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:48.141076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-1c82e987-c61c-4870-9c94-4fc0c173399a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 16:54:48.141076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:48.141076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:48.141076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:48.141076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:48.141076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:48.141076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:48.141076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:48.141076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:48.141076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:48.141076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:48.141076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:48.141076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:48.141076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:48.141076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:48.141076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:48.141938 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:48.141938 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:48.141938 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:48.141938 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:48.141938 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:48.141938 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:48.141938 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:48.141938 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:48.141938 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:48.141938 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:48.141938 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:48.141938 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:48.141938 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:48.141938 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:48.141938 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:48.141938 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:48.148078 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:48.148078 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:48.148078 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:48.148078 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:48.148078 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:48.148078 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:48.148078 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:48.148078 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:48.148078 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:48.148078 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:48.148078 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 16:54:48.148078 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:48.148689 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba030e1-2b93-4689-bc88-428154da8a2e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:48.148689 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba030e1-2b93-4689-bc88-428154da8a2e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:48.148810 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 261/495] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 6 16:54:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:48.148874 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba030e1-2b93-4689-bc88-428154da8a2e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:48.148952 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c82e987-c61c-4870-9c94-4fc0c173399a None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:48.182289 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:48.191135 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 235/496] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Jul 6 16:54:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:48.191135 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe821c-3168-4c57-864d-ee4df17bc673 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:48.191135 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe821c-3168-4c57-864d-ee4df17bc673 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:48.191135 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fe821c-3168-4c57-864d-ee4df17bc673 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:48.191135 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 236/497] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:48.191962 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fe821c-3168-4c57-864d-ee4df17bc673 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:48.258729 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:48.267175 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c82e987-c61c-4870-9c94-4fc0c173399a None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:48.299039 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fe821c-3168-4c57-864d-ee4df17bc673 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:48.303846 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fc183a-2580-4214-ab83-5c439226afaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:48.304522 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc183a-2580-4214-ab83-5c439226afaa None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:48.304810 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc183a-2580-4214-ab83-5c439226afaa None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:48.305091 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fc183a-2580-4214-ab83-5c439226afaa None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:48.305883 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fc183a-2580-4214-ab83-5c439226afaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:48.306226 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fc183a-2580-4214-ab83-5c439226afaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:48.307707 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fc183a-2580-4214-ab83-5c439226afaa None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:48.319698 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 262/498] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:54:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:48.320533 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fc183a-2580-4214-ab83-5c439226afaa None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:48.340411 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669fd6ee-59e3-4dc9-9fea-78390e61282e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:48.341084 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669fd6ee-59e3-4dc9-9fea-78390e61282e None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:48.341375 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669fd6ee-59e3-4dc9-9fea-78390e61282e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:48.341656 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669fd6ee-59e3-4dc9-9fea-78390e61282e None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:48.342403 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669fd6ee-59e3-4dc9-9fea-78390e61282e None admin] RBAC: Adding query filter params (name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:48.342720 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669fd6ee-59e3-4dc9-9fea-78390e61282e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:48.344146 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669fd6ee-59e3-4dc9-9fea-78390e61282e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:48.352451 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 237/499] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 6 16:54:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:48.354187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6bbd70-fd48-4a4f-9f24-ba3819423a0a None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:48.354480 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6bbd70-fd48-4a4f-9f24-ba3819423a0a None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:48.354765 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6bbd70-fd48-4a4f-9f24-ba3819423a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:48.370568 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-0a6bbd70-fd48-4a4f-9f24-ba3819423a0a None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:48.371776 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-0a6bbd70-fd48-4a4f-9f24-ba3819423a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:48.427525 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:48.430716 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fc183a-2580-4214-ab83-5c439226afaa None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:48.437836 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 238/500] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:48.438644 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6bbd70-fd48-4a4f-9f24-ba3819423a0a None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:48.468032 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a797a4-064b-4f29-9985-8b8cd4e8b7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:48.468693 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a797a4-064b-4f29-9985-8b8cd4e8b7af None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:48.468986 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a797a4-064b-4f29-9985-8b8cd4e8b7af None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:48.469270 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a797a4-064b-4f29-9985-8b8cd4e8b7af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:48.492978 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:48.537176 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a797a4-064b-4f29-9985-8b8cd4e8b7af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:48.537464 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6bbd70-fd48-4a4f-9f24-ba3819423a0a None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:48.538564 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a797a4-064b-4f29-9985-8b8cd4e8b7af None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:48.572219 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6575c75-801d-4937-bfc9-7f8e66484a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:48.573013 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6575c75-801d-4937-bfc9-7f8e66484a2a None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:48.573301 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6575c75-801d-4937-bfc9-7f8e66484a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:48.573580 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6575c75-801d-4937-bfc9-7f8e66484a2a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:48.580555 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6575c75-801d-4937-bfc9-7f8e66484a2a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:48.581991 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6575c75-801d-4937-bfc9-7f8e66484a2a None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:48.597998 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-b6575c75-801d-4937-bfc9-7f8e66484a2a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 06 16:54:48.597998 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:48.597998 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:48.597998 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:48.597998 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:48.597998 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:48.597998 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:48.597998 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:48.597998 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:48.597998 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:48.597998 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:48.597998 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:48.597998 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 16:54:48.597998 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 16:54:48.597998 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 16:54:48.597998 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 16:54:48.599079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:48.599079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:48.599079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 16:54:48.599079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:54:48.599079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 16:54:48.599079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:54:48.599079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:48.599079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:48.599079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:48.599079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:48.599079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:48.599079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:48.599079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:48.599079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:48.599079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:48.599079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:48.599079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:48.599079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:48.600037 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:48.600037 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:48.600037 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 16:54:48.600037 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 16:54:48.600037 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 16:54:48.600037 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 16:54:48.600037 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 16:54:48.600037 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 16:54:48.600037 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 06 16:54:48.600037 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:48.600037 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 239/501] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:54:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:48.600595 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6575c75-801d-4937-bfc9-7f8e66484a2a None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:48.636044 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 263/502] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:54:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 16:54:48.637434 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace922e-7bd8-43e1-ab31-242dfb17590d None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:48.637780 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace922e-7bd8-43e1-ab31-242dfb17590d None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:48.638067 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ace922e-7bd8-43e1-ab31-242dfb17590d None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:48.639500 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 264/503] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:48.644197 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace922e-7bd8-43e1-ab31-242dfb17590d None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:48.655562 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:48.700744 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:48.701112 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6575c75-801d-4937-bfc9-7f8e66484a2a None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:48.739778 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64775642-022c-41b2-b130-73e9394b4ee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:48.740583 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64775642-022c-41b2-b130-73e9394b4ee5 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:48.740869 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64775642-022c-41b2-b130-73e9394b4ee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:48.741144 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64775642-022c-41b2-b130-73e9394b4ee5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:48.746714 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ace922e-7bd8-43e1-ab31-242dfb17590d None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:48.751477 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64775642-022c-41b2-b130-73e9394b4ee5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:48.752964 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64775642-022c-41b2-b130-73e9394b4ee5 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:48.769617 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-64775642-022c-41b2-b130-73e9394b4ee5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:48.769617 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:48.769617 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:48.769617 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:48.769617 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:48.769617 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:48.769617 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:48.769617 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:48.769617 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:48.769617 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:48.769617 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:48.769617 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:48.769617 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:54:48.769617 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:54:48.769617 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:54:48.770536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:54:48.770536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:48.770536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:48.770536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:48.770536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:48.770536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:48.770536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:48.770536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:48.770536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:48.770536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:48.770536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:48.770536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:48.770536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:48.770536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:48.770536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:48.770536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:48.770536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:48.771659 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:54:48.771659 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:54:48.771659 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:48.771659 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:48.771659 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 240/504] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:54:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:48.772171 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64775642-022c-41b2-b130-73e9394b4ee5 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:48.783489 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d0556-1a29-4c01-98a0-42d1a93c5ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:48.784313 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d0556-1a29-4c01-98a0-42d1a93c5ade None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:48.784595 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d0556-1a29-4c01-98a0-42d1a93c5ade None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:48.784866 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7d0556-1a29-4c01-98a0-42d1a93c5ade None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:48.795796 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7d0556-1a29-4c01-98a0-42d1a93c5ade None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:48.797149 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7d0556-1a29-4c01-98a0-42d1a93c5ade None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:48.813735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-ff7d0556-1a29-4c01-98a0-42d1a93c5ade None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:48.813735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:48.813735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:48.813735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:48.813735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:48.813735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:48.813735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:48.813735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:48.813735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:48.813735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:48.813735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:48.813735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:48.813735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:54:48.813735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:54:48.813735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:54:48.814554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:54:48.814554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:48.814554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:48.814554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:48.814554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:48.814554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:48.814554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:48.814554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:48.814554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:48.814554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:48.814554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:48.814554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:48.814554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:48.814554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:48.814554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:48.814554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:48.814554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:48.815411 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:54:48.815411 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:54:48.815411 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:48.815411 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:48.815411 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 265/505] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:54:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:48.815669 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d0556-1a29-4c01-98a0-42d1a93c5ade None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:48.870727 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64775642-022c-41b2-b130-73e9394b4ee5 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:48.872991 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:48.905375 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba50d285-8367-4444-bd9d-3819f1c086ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:48.906281 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50d285-8367-4444-bd9d-3819f1c086ef None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:48.906570 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50d285-8367-4444-bd9d-3819f1c086ef None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:48.906855 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba50d285-8367-4444-bd9d-3819f1c086ef None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:48.907722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba50d285-8367-4444-bd9d-3819f1c086ef None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:48.908069 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba50d285-8367-4444-bd9d-3819f1c086ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:48.909867 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba50d285-8367-4444-bd9d-3819f1c086ef None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:48.916823 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7d0556-1a29-4c01-98a0-42d1a93c5ade None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:48.924152 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:48.926949 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 241/506] 10.222.0.40 () {56 vars in 1137 bytes} [Wed Jul 6 16:54:48 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:48.928186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec3c84-fbf8-437e-afb7-520359877a30 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:48.928477 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec3c84-fbf8-437e-afb7-520359877a30 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:48.928756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffec3c84-fbf8-437e-afb7-520359877a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:48.945197 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-ffec3c84-fbf8-437e-afb7-520359877a30 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:48.946961 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ffec3c84-fbf8-437e-afb7-520359877a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:48.951155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d92584b-fa52-4d13-b928-e133132da82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:48.951984 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92584b-fa52-4d13-b928-e133132da82f None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:48.952273 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92584b-fa52-4d13-b928-e133132da82f None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:48.952554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d92584b-fa52-4d13-b928-e133132da82f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:48.963645 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d92584b-fa52-4d13-b928-e133132da82f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:48.965156 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d92584b-fa52-4d13-b928-e133132da82f None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:48.982298 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-4d92584b-fa52-4d13-b928-e133132da82f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:48.982298 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:48.982298 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:48.982298 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:48.982298 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:48.982298 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:48.982298 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:48.982298 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:48.982298 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:48.982298 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:48.982298 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:48.982298 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:48.982298 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:54:48.982298 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:54:48.982298 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:54:48.983125 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:54:48.983125 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:48.983125 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:48.983125 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:48.983125 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:48.983125 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:48.983125 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:48.983125 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:48.983125 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:48.983125 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:48.983125 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:48.983125 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:48.983125 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:48.983125 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:48.983125 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:48.983125 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:48.983125 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:48.983967 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:54:48.983967 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:54:48.983967 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:48.983967 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:48.988004 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 266/507] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:54:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:48.988712 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d92584b-fa52-4d13-b928-e133132da82f None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:49.011760 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 242/508] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:49.012587 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffec3c84-fbf8-437e-afb7-520359877a30 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:49.099021 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d92584b-fa52-4d13-b928-e133132da82f None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:49.101522 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:49.109976 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffec3c84-fbf8-437e-afb7-520359877a30 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:49.132516 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c02283-44a9-4bb5-ad1b-6fb98850787b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:49.133182 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c02283-44a9-4bb5-ad1b-6fb98850787b None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:49.133464 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c02283-44a9-4bb5-ad1b-6fb98850787b None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:49.134266 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c02283-44a9-4bb5-ad1b-6fb98850787b None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:49.135011 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c02283-44a9-4bb5-ad1b-6fb98850787b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:49.135321 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c02283-44a9-4bb5-ad1b-6fb98850787b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:49.136747 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c02283-44a9-4bb5-ad1b-6fb98850787b None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:49.144241 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dfe904-b82e-44ae-aa6f-cdef4dfaf5de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:49.144906 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dfe904-b82e-44ae-aa6f-cdef4dfaf5de None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:49.145189 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dfe904-b82e-44ae-aa6f-cdef4dfaf5de None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:49.145467 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dfe904-b82e-44ae-aa6f-cdef4dfaf5de None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:49.146453 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dfe904-b82e-44ae-aa6f-cdef4dfaf5de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:49.146761 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dfe904-b82e-44ae-aa6f-cdef4dfaf5de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:49.148092 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 267/509] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:54:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:49.148195 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dfe904-b82e-44ae-aa6f-cdef4dfaf5de None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:49.148973 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c02283-44a9-4bb5-ad1b-6fb98850787b None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:49.160654 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 243/510] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:54:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:49.161459 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dfe904-b82e-44ae-aa6f-cdef4dfaf5de None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:49.173269 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:49.253223 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c02283-44a9-4bb5-ad1b-6fb98850787b None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:49.260574 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dfe904-b82e-44ae-aa6f-cdef4dfaf5de None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:49.286847 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0f441d-e3aa-4b9a-bce1-7d0972570c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:49.287496 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f441d-e3aa-4b9a-bce1-7d0972570c01 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:49.287809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f441d-e3aa-4b9a-bce1-7d0972570c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:49.288085 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f441d-e3aa-4b9a-bce1-7d0972570c01 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:49.294060 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22491c26-333f-4412-a2d9-1ec3af6b58f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:49.294718 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22491c26-333f-4412-a2d9-1ec3af6b58f0 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:49.295008 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22491c26-333f-4412-a2d9-1ec3af6b58f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:49.295292 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22491c26-333f-4412-a2d9-1ec3af6b58f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:49.296096 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22491c26-333f-4412-a2d9-1ec3af6b58f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:49.296810 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22491c26-333f-4412-a2d9-1ec3af6b58f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:49.298206 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22491c26-333f-4412-a2d9-1ec3af6b58f0 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:49.311184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 244/511] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:54:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:49.312009 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22491c26-333f-4412-a2d9-1ec3af6b58f0 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:49.327991 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:49.355542 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0f441d-e3aa-4b9a-bce1-7d0972570c01 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:49.356982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0f441d-e3aa-4b9a-bce1-7d0972570c01 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:49.374207 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:49.412829 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22491c26-333f-4412-a2d9-1ec3af6b58f0 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:49.447415 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d8a9d-3af8-4a84-87b6-c06ab80f9e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:49.448309 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d8a9d-3af8-4a84-87b6-c06ab80f9e56 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:49.448600 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d8a9d-3af8-4a84-87b6-c06ab80f9e56 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:49.448874 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7d8a9d-3af8-4a84-87b6-c06ab80f9e56 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:49.452599 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 268/512] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:54:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 16:54:49.453426 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0f441d-e3aa-4b9a-bce1-7d0972570c01 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:49.459165 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7d8a9d-3af8-4a84-87b6-c06ab80f9e56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:49.467477 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-5f7d8a9d-3af8-4a84-87b6-c06ab80f9e56 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 16:54:49.467477 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:49.467477 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:49.467477 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:49.467477 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:49.467477 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:49.467477 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:49.467477 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:49.467477 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:49.467477 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:49.467477 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:49.467477 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:49.467477 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:49.467477 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:49.467477 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:49.467477 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:49.468421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:49.468421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:49.468421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:49.468421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:49.468421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:49.468421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:49.468421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:49.468421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:49.468421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:49.468421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:49.468421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:49.468421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:49.468421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:49.468421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:49.468421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:49.468421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:49.469226 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:49.469226 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:49.469226 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:49.469226 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:49.469226 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:49.469226 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:49.469226 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:49.469226 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:49.469226 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:49.469226 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:49.469226 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 16:54:49.469226 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:49.472021 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 245/513] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 6 16:54:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:49.472682 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d8a9d-3af8-4a84-87b6-c06ab80f9e56 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:49.541891 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:49.552233 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0f441d-e3aa-4b9a-bce1-7d0972570c01 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:49.569142 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:49.569815 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7d8a9d-3af8-4a84-87b6-c06ab80f9e56 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:49.588595 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7967e4ce-b503-4735-89b6-f12ba95bb37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:49.589401 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967e4ce-b503-4735-89b6-f12ba95bb37b None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:49.589688 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967e4ce-b503-4735-89b6-f12ba95bb37b None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:49.589964 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7967e4ce-b503-4735-89b6-f12ba95bb37b None admin] PATH_INFO: `/v3/users/glance` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:49.597514 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7967e4ce-b503-4735-89b6-f12ba95bb37b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:49.598972 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7967e4ce-b503-4735-89b6-f12ba95bb37b None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:49.605067 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bde7edb-476f-4b37-840a-f9481a0b1bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:49.605804 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde7edb-476f-4b37-840a-f9481a0b1bdc None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:49.606035 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde7edb-476f-4b37-840a-f9481a0b1bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:49.606311 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bde7edb-476f-4b37-840a-f9481a0b1bdc None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:49.607015 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bde7edb-476f-4b37-840a-f9481a0b1bdc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:49.608463 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bde7edb-476f-4b37-840a-f9481a0b1bdc None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:49.614278 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-7967e4ce-b503-4735-89b6-f12ba95bb37b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 06 16:54:49.614278 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:49.614278 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:49.614278 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:49.614278 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:49.614278 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:49.614278 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:49.614278 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:49.614278 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:49.614278 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:49.614278 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:49.614278 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:49.614278 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 16:54:49.614278 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 16:54:49.614278 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 16:54:49.614278 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 16:54:49.615199 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:49.615199 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:49.615199 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 16:54:49.615199 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:54:49.615199 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 16:54:49.615199 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:54:49.615199 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:49.615199 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:49.615199 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:49.615199 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:49.615199 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:49.615199 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:49.615199 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:49.615199 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:49.615199 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:49.615199 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:49.615199 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:49.615199 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:49.616122 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:49.616122 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:49.616122 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 16:54:49.616122 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 16:54:49.616122 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 16:54:49.616122 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 16:54:49.616122 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 16:54:49.616122 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 16:54:49.616122 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 06 16:54:49.616122 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:49.616122 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 269/514] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:54:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:49.616678 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7967e4ce-b503-4735-89b6-f12ba95bb37b None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:49.696980 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 246/515] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 6 16:54:49 2022] POST /identity/v3/users => generated 247 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 16:54:49.697809 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bde7edb-476f-4b37-840a-f9481a0b1bdc None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:49.715820 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7967e4ce-b503-4735-89b6-f12ba95bb37b None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:49.764008 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7251caf-9062-4760-af14-3e5376162687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:49.765725 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7251caf-9062-4760-af14-3e5376162687 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:49.766014 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7251caf-9062-4760-af14-3e5376162687 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:49.766293 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7251caf-9062-4760-af14-3e5376162687 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:49.775435 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7251caf-9062-4760-af14-3e5376162687 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:49.776968 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7251caf-9062-4760-af14-3e5376162687 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:49.793310 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-f7251caf-9062-4760-af14-3e5376162687 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:54:49.793310 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:49.793310 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:49.793310 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:49.793310 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:49.793310 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:49.793310 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:49.793310 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:49.793310 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:49.793310 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:49.793310 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:49.793310 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:49.793310 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 16:54:49.793310 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 16:54:49.793310 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 16:54:49.794245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 16:54:49.794245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:49.794245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:49.794245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:49.794245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:49.794245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:49.794245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:49.794245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:49.794245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:49.794245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:49.794245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:49.794245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:49.794245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:49.794245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:49.794245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:49.794245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:49.794245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:49.795085 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 16:54:49.795085 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 16:54:49.795085 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 16:54:49.795085 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 16:54:49.795085 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 16:54:49.795085 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 16:54:49.795085 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:54:49.795085 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:49.795085 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 270/516] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:54:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:49.795537 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7251caf-9062-4760-af14-3e5376162687 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:49.841532 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bde7edb-476f-4b37-840a-f9481a0b1bdc None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:49.842824 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:49.871922 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:49.879915 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64681e0b-9502-430a-ade4-ee9264637134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:49.881361 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64681e0b-9502-430a-ade4-ee9264637134 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:49.881648 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64681e0b-9502-430a-ade4-ee9264637134 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:49.902944 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64681e0b-9502-430a-ade4-ee9264637134 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:49.902944 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64681e0b-9502-430a-ade4-ee9264637134 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:49.902944 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64681e0b-9502-430a-ade4-ee9264637134 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:49.913293 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7251caf-9062-4760-af14-3e5376162687 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:49.915041 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-64681e0b-9502-430a-ade4-ee9264637134 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:49.915041 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:49.915041 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:49.915041 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:49.915041 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:49.915041 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:49.915041 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:49.915041 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:49.915041 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:49.915041 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:49.915041 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:49.915041 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:49.915041 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:54:49.915041 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:54:49.915041 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:54:49.916082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:54:49.916082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:49.916082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:49.916082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:49.916082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:49.916082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:49.916082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:49.916082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:49.916082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:49.916082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:49.916082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:49.916082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:49.916082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:49.916082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:49.916082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:49.916082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:49.916082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:49.917062 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:54:49.917062 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:54:49.917062 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:49.917062 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:49.917062 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 247/517] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:54:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:49.917427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64681e0b-9502-430a-ade4-ee9264637134 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:49.951624 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77576f43-727e-4378-ad3e-b0e30732db46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:49.952598 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77576f43-727e-4378-ad3e-b0e30732db46 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:49.952885 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77576f43-727e-4378-ad3e-b0e30732db46 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:49.953192 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77576f43-727e-4378-ad3e-b0e30732db46 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:49.953948 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77576f43-727e-4378-ad3e-b0e30732db46 None admin] RBAC: Adding query filter params (name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:49.954250 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77576f43-727e-4378-ad3e-b0e30732db46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:49.955613 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77576f43-727e-4378-ad3e-b0e30732db46 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:49.964736 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 271/518] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 6 16:54:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:49.965549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77576f43-727e-4378-ad3e-b0e30732db46 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:50.029243 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64681e0b-9502-430a-ade4-ee9264637134 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:50.062589 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a2eb75-76b5-473c-89fc-6ed66cea38b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:50.063568 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2eb75-76b5-473c-89fc-6ed66cea38b1 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:50.063893 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2eb75-76b5-473c-89fc-6ed66cea38b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:50.064176 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2eb75-76b5-473c-89fc-6ed66cea38b1 None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:50.065027 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a2eb75-76b5-473c-89fc-6ed66cea38b1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:50.065341 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a2eb75-76b5-473c-89fc-6ed66cea38b1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:50.066790 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a2eb75-76b5-473c-89fc-6ed66cea38b1 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:50.071323 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:50.075194 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77576f43-727e-4378-ad3e-b0e30732db46 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:50.084639 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 248/519] 10.222.0.40 () {56 vars in 1137 bytes} [Wed Jul 6 16:54:49 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:50.085474 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a2eb75-76b5-473c-89fc-6ed66cea38b1 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:50.121154 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d62c4b-c880-4753-a515-9311f0f5239b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:50.121823 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d62c4b-c880-4753-a515-9311f0f5239b None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:50.122203 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d62c4b-c880-4753-a515-9311f0f5239b None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:50.122539 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d62c4b-c880-4753-a515-9311f0f5239b None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:50.123462 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d62c4b-c880-4753-a515-9311f0f5239b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:50.123655 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d62c4b-c880-4753-a515-9311f0f5239b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:50.125083 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d62c4b-c880-4753-a515-9311f0f5239b None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:50.129682 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:50.143323 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 272/520] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 6 16:54:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:50.144074 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d62c4b-c880-4753-a515-9311f0f5239b None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:50.193363 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a2eb75-76b5-473c-89fc-6ed66cea38b1 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:50.229719 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf54640-232a-4f6f-a350-d323fa318809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:50.230380 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf54640-232a-4f6f-a350-d323fa318809 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:50.230666 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf54640-232a-4f6f-a350-d323fa318809 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:50.230933 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf54640-232a-4f6f-a350-d323fa318809 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:50.231657 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf54640-232a-4f6f-a350-d323fa318809 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:50.231992 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf54640-232a-4f6f-a350-d323fa318809 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:50.233344 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf54640-232a-4f6f-a350-d323fa318809 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:50.249734 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 249/521] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:54:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:50.249734 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf54640-232a-4f6f-a350-d323fa318809 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:50.263587 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d62c4b-c880-4753-a515-9311f0f5239b None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:50.299943 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6263efb-539d-4411-9a60-80c4b5f9d71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:50.300744 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6263efb-539d-4411-9a60-80c4b5f9d71e None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:50.301029 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6263efb-539d-4411-9a60-80c4b5f9d71e None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:50.301307 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6263efb-539d-4411-9a60-80c4b5f9d71e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:50.313047 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6263efb-539d-4411-9a60-80c4b5f9d71e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:50.314489 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6263efb-539d-4411-9a60-80c4b5f9d71e None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:50.333111 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-e6263efb-539d-4411-9a60-80c4b5f9d71e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:50.333111 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:50.333111 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:50.333111 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:50.333111 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:50.333111 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:50.333111 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:50.333111 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:50.333111 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:50.333111 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:50.333111 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:50.333111 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:50.333111 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:54:50.333111 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:54:50.333111 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:54:50.333948 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:54:50.333948 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:50.333948 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:50.333948 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:50.333948 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:50.333948 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:50.333948 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:50.333948 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:50.333948 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:50.333948 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:50.333948 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:50.333948 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:50.333948 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:50.333948 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:50.333948 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:50.333948 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:50.333948 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:50.334955 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:54:50.334955 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:54:50.334955 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:50.334955 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:50.334955 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 273/522] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:54:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:50.335837 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6263efb-539d-4411-9a60-80c4b5f9d71e None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:50.353487 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:50.354911 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:50.358573 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf54640-232a-4f6f-a350-d323fa318809 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:50.391799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf556b-cf89-4ecb-9073-050f3ad407c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:50.392395 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf556b-cf89-4ecb-9073-050f3ad407c5 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:50.392676 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf556b-cf89-4ecb-9073-050f3ad407c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:50.392945 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cf556b-cf89-4ecb-9073-050f3ad407c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:50.441832 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6263efb-539d-4411-9a60-80c4b5f9d71e None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:50.463648 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cf556b-cf89-4ecb-9073-050f3ad407c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:50.465126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cf556b-cf89-4ecb-9073-050f3ad407c5 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:50.477151 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93402c68-cfae-470f-ad25-8da4b9c568d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:50.478340 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93402c68-cfae-470f-ad25-8da4b9c568d2 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:50.478623 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93402c68-cfae-470f-ad25-8da4b9c568d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:50.478905 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93402c68-cfae-470f-ad25-8da4b9c568d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:50.479751 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93402c68-cfae-470f-ad25-8da4b9c568d2 None admin] RBAC: Adding query filter params (role.id=750bca9c32ac4aa49adcbee30463f1fa, scope.project.id=7938b0346d4240ab89774327d09a89ae, user.id=af26ef96dc9d47dc91549b736939268e) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:50.480092 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93402c68-cfae-470f-ad25-8da4b9c568d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:50.481536 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93402c68-cfae-470f-ad25-8da4b9c568d2 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:50.487493 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 274/523] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:54:50 2022] GET /identity/v3/role_assignments?user.id=af26ef96dc9d47dc91549b736939268e&scope.project.id=7938b0346d4240ab89774327d09a89ae&role.id=750bca9c32ac4aa49adcbee30463f1fa => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:50.492718 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93402c68-cfae-470f-ad25-8da4b9c568d2 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:50.593091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:50.602343 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:50.602941 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 250/524] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:54:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 16:54:50.604109 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf556b-cf89-4ecb-9073-050f3ad407c5 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:50.631577 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93402c68-cfae-470f-ad25-8da4b9c568d2 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:50.680589 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9129fb1b-d0df-4835-9992-49e9c603613e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:50.681984 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129fb1b-d0df-4835-9992-49e9c603613e None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:50.682692 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129fb1b-d0df-4835-9992-49e9c603613e None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:50.683140 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9129fb1b-d0df-4835-9992-49e9c603613e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:50.692030 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9129fb1b-d0df-4835-9992-49e9c603613e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:50.693675 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9129fb1b-d0df-4835-9992-49e9c603613e None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:50.712760 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-9129fb1b-d0df-4835-9992-49e9c603613e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:54:50.712760 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:50.712760 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:50.712760 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:50.712760 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:50.712760 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:50.712760 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:50.712760 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:50.712760 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:50.712760 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:50.712760 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:50.712760 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:50.712760 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 16:54:50.712760 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 16:54:50.712760 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 16:54:50.713813 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 16:54:50.713813 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:50.713813 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:50.713813 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:50.713813 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:50.713813 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:50.713813 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:50.713813 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:50.713813 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:50.713813 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:50.713813 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:50.713813 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:50.713813 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:50.713813 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:50.713813 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:50.713813 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:50.713813 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:50.714881 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 16:54:50.714881 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 16:54:50.714881 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 16:54:50.714881 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 16:54:50.714881 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 16:54:50.714881 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 16:54:50.714881 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:54:50.714881 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:50.721676 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 275/525] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:54:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:50.721676 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9129fb1b-d0df-4835-9992-49e9c603613e None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:50.749376 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cf556b-cf89-4ecb-9073-050f3ad407c5 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:50.793764 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d5ae2-af3b-4b78-8eb8-491bcb2b5c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:50.794459 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d5ae2-af3b-4b78-8eb8-491bcb2b5c2c None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:50.795273 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d5ae2-af3b-4b78-8eb8-491bcb2b5c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:50.795556 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54d5ae2-af3b-4b78-8eb8-491bcb2b5c2c None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:50.796586 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54d5ae2-af3b-4b78-8eb8-491bcb2b5c2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:50.796895 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54d5ae2-af3b-4b78-8eb8-491bcb2b5c2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:50.798970 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54d5ae2-af3b-4b78-8eb8-491bcb2b5c2c None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:50.814788 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 251/526] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:54:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:50.816551 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d5ae2-af3b-4b78-8eb8-491bcb2b5c2c None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:50.857838 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9129fb1b-d0df-4835-9992-49e9c603613e None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:50.895508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:50.900018 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5868a56c-41b1-4689-b9e9-d21b9341db9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:50.900883 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5868a56c-41b1-4689-b9e9-d21b9341db9f None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:50.901171 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5868a56c-41b1-4689-b9e9-d21b9341db9f None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:50.901451 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5868a56c-41b1-4689-b9e9-d21b9341db9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:50.912089 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:50.913301 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5868a56c-41b1-4689-b9e9-d21b9341db9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:50.915415 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5868a56c-41b1-4689-b9e9-d21b9341db9f None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:50.937017 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-5868a56c-41b1-4689-b9e9-d21b9341db9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:50.937017 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:50.937017 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:50.937017 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:50.937017 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:50.937017 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:50.937017 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:50.937017 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:50.937017 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:50.937017 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:50.937017 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:50.937017 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:50.937017 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:54:50.937017 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:54:50.937017 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:54:50.937915 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:54:50.937915 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:50.937915 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:50.937915 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:50.937915 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:50.937915 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:50.937915 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:50.937915 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:50.937915 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:50.937915 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:50.937915 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:50.937915 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:50.937915 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:50.937915 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:50.937915 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:50.937915 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:50.937915 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:50.938740 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:54:50.938740 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:54:50.938740 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:50.938740 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:50.938966 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 276/527] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:54:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:50.942285 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5868a56c-41b1-4689-b9e9-d21b9341db9f None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:50.947217 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54d5ae2-af3b-4b78-8eb8-491bcb2b5c2c None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:50.988925 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ef17a-321c-4315-b33c-a8de1e690e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:50.989745 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ef17a-321c-4315-b33c-a8de1e690e5b None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:50.990026 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ef17a-321c-4315-b33c-a8de1e690e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:50.990662 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ef17a-321c-4315-b33c-a8de1e690e5b None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:50.991714 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ef17a-321c-4315-b33c-a8de1e690e5b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:50.992053 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ef17a-321c-4315-b33c-a8de1e690e5b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:50.994180 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ef17a-321c-4315-b33c-a8de1e690e5b None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:51.014735 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 252/528] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 6 16:54:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:51.015740 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ef17a-321c-4315-b33c-a8de1e690e5b None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:51.053538 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5868a56c-41b1-4689-b9e9-d21b9341db9f None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:51.089553 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14557712-480f-414b-aa91-3f15de34ef33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:51.090658 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14557712-480f-414b-aa91-3f15de34ef33 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:51.091074 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14557712-480f-414b-aa91-3f15de34ef33 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:51.091453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14557712-480f-414b-aa91-3f15de34ef33 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:51.099968 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14557712-480f-414b-aa91-3f15de34ef33 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:51.101658 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14557712-480f-414b-aa91-3f15de34ef33 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:51.117926 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-14557712-480f-414b-aa91-3f15de34ef33 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 06 16:54:51.117926 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:51.117926 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:51.117926 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:51.117926 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:51.117926 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:51.117926 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:51.117926 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:51.117926 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:51.117926 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:51.117926 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:51.117926 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:51.117926 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 16:54:51.117926 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 16:54:51.117926 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 16:54:51.117926 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 16:54:51.120308 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:51.120308 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:51.120308 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 16:54:51.120308 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:54:51.120308 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 16:54:51.120308 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:54:51.120308 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:51.120308 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:51.120308 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:51.120308 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:51.120308 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:51.120308 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:51.120308 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:51.120308 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:51.120308 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:51.120308 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:51.120308 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:51.120308 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:51.122372 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:51.122372 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:51.122372 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 16:54:51.122372 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 16:54:51.122372 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 16:54:51.122372 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 16:54:51.122372 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 16:54:51.122372 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 16:54:51.122372 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 06 16:54:51.122372 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:51.122372 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 277/529] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 6 16:54:50 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:51.122372 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14557712-480f-414b-aa91-3f15de34ef33 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:51.136331 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ef17a-321c-4315-b33c-a8de1e690e5b None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:51.158775 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:51.177850 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:51.180740 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56c16c-d8b3-4785-af26-22ccb4ef6302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:51.181584 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56c16c-d8b3-4785-af26-22ccb4ef6302 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:51.181735 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56c16c-d8b3-4785-af26-22ccb4ef6302 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:51.181941 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e56c16c-d8b3-4785-af26-22ccb4ef6302 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:51.183109 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56c16c-d8b3-4785-af26-22ccb4ef6302 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:51.183415 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56c16c-d8b3-4785-af26-22ccb4ef6302 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:51.185142 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e56c16c-d8b3-4785-af26-22ccb4ef6302 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:51.204119 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 253/530] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:54:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:51.204769 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56c16c-d8b3-4785-af26-22ccb4ef6302 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:51.236961 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14557712-480f-414b-aa91-3f15de34ef33 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:51.276552 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7bd652-a5b9-4585-a74f-924fd739b36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:51.277279 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7bd652-a5b9-4585-a74f-924fd739b36b None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:51.277635 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7bd652-a5b9-4585-a74f-924fd739b36b None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:51.277917 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7bd652-a5b9-4585-a74f-924fd739b36b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:51.278820 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7bd652-a5b9-4585-a74f-924fd739b36b None admin] RBAC: Adding query filter params (role.id=750bca9c32ac4aa49adcbee30463f1fa, scope.project.id=7938b0346d4240ab89774327d09a89ae, user.id=a1bfa3e856ae46d48c82195f70aec24f) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:51.279139 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7bd652-a5b9-4585-a74f-924fd739b36b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:51.280758 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7bd652-a5b9-4585-a74f-924fd739b36b None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:51.288928 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 278/531] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:54:51 2022] GET /identity/v3/role_assignments?user.id=a1bfa3e856ae46d48c82195f70aec24f&scope.project.id=7938b0346d4240ab89774327d09a89ae&role.id=750bca9c32ac4aa49adcbee30463f1fa => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:51.289801 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7bd652-a5b9-4585-a74f-924fd739b36b None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:51.337937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e56c16c-d8b3-4785-af26-22ccb4ef6302 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:51.380882 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9849dfd-87b1-48a0-8142-1744328c5afd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:51.382169 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9849dfd-87b1-48a0-8142-1744328c5afd None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:51.382455 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9849dfd-87b1-48a0-8142-1744328c5afd None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:51.382731 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9849dfd-87b1-48a0-8142-1744328c5afd None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:51.384410 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9849dfd-87b1-48a0-8142-1744328c5afd None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:51.384719 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9849dfd-87b1-48a0-8142-1744328c5afd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:51.387270 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9849dfd-87b1-48a0-8142-1744328c5afd None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:51.412166 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 254/532] 10.222.0.40 () {56 vars in 1139 bytes} [Wed Jul 6 16:54:51 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:51.417556 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9849dfd-87b1-48a0-8142-1744328c5afd None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:51.424728 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7bd652-a5b9-4585-a74f-924fd739b36b None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:51.443095 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:51.456803 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:51.474705 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4507d4-17a7-4133-ad0c-5cf7dbdc6119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:51.475526 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4507d4-17a7-4133-ad0c-5cf7dbdc6119 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:51.475836 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4507d4-17a7-4133-ad0c-5cf7dbdc6119 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:51.476112 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4507d4-17a7-4133-ad0c-5cf7dbdc6119 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:51.486334 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4507d4-17a7-4133-ad0c-5cf7dbdc6119 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:51.488283 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4507d4-17a7-4133-ad0c-5cf7dbdc6119 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:51.507340 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-0d4507d4-17a7-4133-ad0c-5cf7dbdc6119 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:54:51.507340 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:51.507340 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:51.507340 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:51.507340 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:51.507340 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:51.507340 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:51.507340 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:51.507340 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:51.507340 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:51.507340 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:51.507340 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:51.507340 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 16:54:51.507340 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 16:54:51.507340 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 16:54:51.510097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 16:54:51.510097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:51.510097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:51.510097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:51.510097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:51.510097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:51.510097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:51.510097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:51.510097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:51.510097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:51.510097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:51.510097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:51.510097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:51.510097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:51.510097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:51.510097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:51.510097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:51.512081 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 16:54:51.512081 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 16:54:51.512081 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 16:54:51.512081 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 16:54:51.512081 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 16:54:51.512081 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 16:54:51.512081 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:54:51.512081 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:51.512081 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 279/533] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:54:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:51.525151 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4507d4-17a7-4133-ad0c-5cf7dbdc6119 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:51.537795 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9849dfd-87b1-48a0-8142-1744328c5afd None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:51.579767 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae86a8-14af-4da4-809a-ea0c0f8b7935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:51.580705 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae86a8-14af-4da4-809a-ea0c0f8b7935 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:51.580992 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae86a8-14af-4da4-809a-ea0c0f8b7935 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:51.581269 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ae86a8-14af-4da4-809a-ea0c0f8b7935 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:51.669486 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ae86a8-14af-4da4-809a-ea0c0f8b7935 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:51.671281 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ae86a8-14af-4da4-809a-ea0c0f8b7935 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:51.732230 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:51.739752 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4507d4-17a7-4133-ad0c-5cf7dbdc6119 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:51.782727 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 255/534] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:54:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 16:54:51.789444 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae86a8-14af-4da4-809a-ea0c0f8b7935 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:51.817633 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79aa72a-32ca-4520-8c1c-cd86d581a55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:51.818601 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79aa72a-32ca-4520-8c1c-cd86d581a55a None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:51.819866 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79aa72a-32ca-4520-8c1c-cd86d581a55a None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:51.820486 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79aa72a-32ca-4520-8c1c-cd86d581a55a None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:51.823213 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79aa72a-32ca-4520-8c1c-cd86d581a55a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:51.823718 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79aa72a-32ca-4520-8c1c-cd86d581a55a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:51.825363 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79aa72a-32ca-4520-8c1c-cd86d581a55a None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:51.850296 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 280/535] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 6 16:54:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:51.855557 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79aa72a-32ca-4520-8c1c-cd86d581a55a None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:51.913545 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:51.995010 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ae86a8-14af-4da4-809a-ea0c0f8b7935 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:52.031258 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79aa72a-32ca-4520-8c1c-cd86d581a55a None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:52.040443 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92627efb-b2b4-4759-9652-69c349821cd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:52.042012 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92627efb-b2b4-4759-9652-69c349821cd1 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:52.042486 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92627efb-b2b4-4759-9652-69c349821cd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:52.043401 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92627efb-b2b4-4759-9652-69c349821cd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:52.056442 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:52.059490 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92627efb-b2b4-4759-9652-69c349821cd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:52.061466 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92627efb-b2b4-4759-9652-69c349821cd1 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:52.075601 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91cf3a0-102d-4a7c-88c0-c6cc59499ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:52.076475 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91cf3a0-102d-4a7c-88c0-c6cc59499ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:52.076765 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91cf3a0-102d-4a7c-88c0-c6cc59499ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:52.077044 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91cf3a0-102d-4a7c-88c0-c6cc59499ed0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:52.085623 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91cf3a0-102d-4a7c-88c0-c6cc59499ed0 None admin] RBAC: Adding query filter params (role.id=750bca9c32ac4aa49adcbee30463f1fa, scope.project.id=7938b0346d4240ab89774327d09a89ae, user.id=635a410f471c43109438d0c39874070b) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:52.085623 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91cf3a0-102d-4a7c-88c0-c6cc59499ed0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:52.085623 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91cf3a0-102d-4a7c-88c0-c6cc59499ed0 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:52.085861 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-92627efb-b2b4-4759-9652-69c349821cd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:52.085861 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:52.085861 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:52.085861 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:52.085861 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:52.085861 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:52.085861 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:52.085861 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:52.085861 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:52.085861 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:52.085861 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:52.085861 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:52.085861 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:54:52.085861 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:54:52.085861 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:54:52.086614 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:54:52.086614 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:52.086614 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:52.086614 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:52.086614 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:52.086614 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:52.086614 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:52.086614 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:52.086614 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:52.086614 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:52.086614 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:52.086614 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:52.086614 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:52.086614 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:52.086614 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:52.086614 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:52.086614 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:52.087468 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:54:52.087468 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:54:52.087468 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:52.087468 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:52.087688 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 281/537] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:54:51 2022] GET /identity/v3/role_assignments?user.id=635a410f471c43109438d0c39874070b&scope.project.id=7938b0346d4240ab89774327d09a89ae&role.id=750bca9c32ac4aa49adcbee30463f1fa => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:52.087688 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee5ab10-9423-4541-be91-4e57a4e2ac81 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:52.087688 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee5ab10-9423-4541-be91-4e57a4e2ac81 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:52.087688 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee5ab10-9423-4541-be91-4e57a4e2ac81 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:52.088329 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 256/537] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:54:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:52.093678 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 282/538] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:52.093860 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92627efb-b2b4-4759-9652-69c349821cd1 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:52.095252 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767d482-a9db-473e-ad9b-db58f300c6e1 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:52.095545 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767d482-a9db-473e-ad9b-db58f300c6e1 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:52.095863 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5767d482-a9db-473e-ad9b-db58f300c6e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:52.117122 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-5767d482-a9db-473e-ad9b-db58f300c6e1 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:52.119057 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-5767d482-a9db-473e-ad9b-db58f300c6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:52.266811 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 283/539] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 16:54:52.287677 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92627efb-b2b4-4759-9652-69c349821cd1 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:52.298334 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5767d482-a9db-473e-ad9b-db58f300c6e1 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:52.337247 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:52.341938 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbfee18-1308-431c-9e48-96af1bc1c69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:52.342611 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbfee18-1308-431c-9e48-96af1bc1c69b None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:52.342896 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbfee18-1308-431c-9e48-96af1bc1c69b None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:52.343175 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbfee18-1308-431c-9e48-96af1bc1c69b None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:52.343952 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbfee18-1308-431c-9e48-96af1bc1c69b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:52.344243 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbfee18-1308-431c-9e48-96af1bc1c69b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:52.345753 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbfee18-1308-431c-9e48-96af1bc1c69b None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:52.371231 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 257/540] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:54:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:52.377633 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbfee18-1308-431c-9e48-96af1bc1c69b None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:52.429394 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5767d482-a9db-473e-ad9b-db58f300c6e1 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:52.469802 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00fa895-60c7-4657-9cd1-480b5d3a7e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:52.471017 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00fa895-60c7-4657-9cd1-480b5d3a7e93 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:52.471555 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00fa895-60c7-4657-9cd1-480b5d3a7e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:52.472226 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00fa895-60c7-4657-9cd1-480b5d3a7e93 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:52.485408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00fa895-60c7-4657-9cd1-480b5d3a7e93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:52.495555 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-a00fa895-60c7-4657-9cd1-480b5d3a7e93 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 16:54:52.495555 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:52.495555 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:52.495555 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:52.495555 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:52.495555 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:52.495555 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:52.495555 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:52.495555 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:52.495555 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:52.495555 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:52.495555 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:52.495555 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:52.495555 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:52.495555 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:52.495555 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:52.496495 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:52.496495 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:52.496495 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:52.496495 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:52.496495 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:52.496495 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:52.496495 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:52.496495 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:52.496495 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:52.496495 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:52.496495 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:52.496495 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:52.496495 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:52.496495 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:52.496495 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:52.496495 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:52.497309 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:52.497309 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:52.497309 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:52.497309 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:52.497309 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:52.497309 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:52.497309 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:52.497309 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:52.497309 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:52.497309 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:52.497309 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 16:54:52.497309 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:52.500403 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 284/541] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 6 16:54:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:52.506540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:52.508047 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00fa895-60c7-4657-9cd1-480b5d3a7e93 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:52.610698 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:52.629252 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00fa895-60c7-4657-9cd1-480b5d3a7e93 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:52.650988 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbfee18-1308-431c-9e48-96af1bc1c69b None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:52.673402 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46f34e-0979-491e-800e-4b011bee752f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:52.675078 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46f34e-0979-491e-800e-4b011bee752f None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:52.675593 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46f34e-0979-491e-800e-4b011bee752f None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:52.676551 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46f34e-0979-491e-800e-4b011bee752f None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:52.678136 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa46f34e-0979-491e-800e-4b011bee752f None admin] RBAC: Adding query filter params (name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:52.678997 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa46f34e-0979-491e-800e-4b011bee752f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:52.681020 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa46f34e-0979-491e-800e-4b011bee752f None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:52.696623 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 285/542] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 6 16:54:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:52.698290 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64a140-4c88-4dfb-895b-7b8364c21df7 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:52.698730 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64a140-4c88-4dfb-895b-7b8364c21df7 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:52.699158 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64a140-4c88-4dfb-895b-7b8364c21df7 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:52.700919 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 286/543] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:52.703182 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64a140-4c88-4dfb-895b-7b8364c21df7 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:52.709544 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bec040-d34d-4b87-acc4-42b612d2d967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:52.710463 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bec040-d34d-4b87-acc4-42b612d2d967 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:52.710884 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bec040-d34d-4b87-acc4-42b612d2d967 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:52.711265 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bec040-d34d-4b87-acc4-42b612d2d967 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:52.719656 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bec040-d34d-4b87-acc4-42b612d2d967 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:52.721298 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bec040-d34d-4b87-acc4-42b612d2d967 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:52.739054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-98bec040-d34d-4b87-acc4-42b612d2d967 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:54:52.739054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:52.739054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:52.739054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:52.739054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:52.739054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:52.739054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:52.739054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:52.739054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:52.739054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:52.739054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:52.739054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:52.739054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 16:54:52.739054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 16:54:52.739054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 16:54:52.740079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 16:54:52.740079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:52.740079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:52.740079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:52.740079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:52.740079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:52.740079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:52.740079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:52.740079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:52.740079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:52.740079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:52.740079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:52.740079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:52.740079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:52.740079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:52.740079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:52.740079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:52.740907 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 16:54:52.740907 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 16:54:52.740907 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 16:54:52.740907 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 16:54:52.740907 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 16:54:52.740907 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 16:54:52.740907 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:54:52.740907 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:52.743804 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 258/544] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:54:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:52.745259 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a53bd1-fa9d-4e08-8a28-a980656b61d5 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:52.745740 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a53bd1-fa9d-4e08-8a28-a980656b61d5 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:52.746187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a53bd1-fa9d-4e08-8a28-a980656b61d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:52.767914 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-95a53bd1-fa9d-4e08-8a28-a980656b61d5 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:52.769423 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-95a53bd1-fa9d-4e08-8a28-a980656b61d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:52.833970 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64a140-4c88-4dfb-895b-7b8364c21df7 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:52.854974 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 259/545] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:52.856085 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a53bd1-fa9d-4e08-8a28-a980656b61d5 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:52.886031 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8d4607-f1a9-496d-9516-9de18830abdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:52.887158 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d4607-f1a9-496d-9516-9de18830abdc None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:52.887667 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d4607-f1a9-496d-9516-9de18830abdc None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:52.888196 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d4607-f1a9-496d-9516-9de18830abdc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:52.899873 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8d4607-f1a9-496d-9516-9de18830abdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:52.901576 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8d4607-f1a9-496d-9516-9de18830abdc None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:52.908964 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:52.910869 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:52.920234 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-0a8d4607-f1a9-496d-9516-9de18830abdc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:52.920234 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:52.920234 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:52.920234 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:52.920234 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:52.920234 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:52.920234 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:52.920234 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:52.920234 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:52.920234 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:52.920234 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:52.920234 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:52.920234 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:54:52.920234 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:54:52.920234 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:54:52.928233 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:54:52.928233 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:52.928233 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:52.928233 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:52.928233 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:52.928233 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:52.928233 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:52.928233 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:52.928233 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:52.928233 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:52.928233 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:52.928233 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:52.928233 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:52.928233 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:52.928233 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:52.928233 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:52.928233 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:52.929069 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:54:52.929069 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:54:52.929069 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:52.929069 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:52.929069 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 287/546] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:54:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:54:52.929069 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8d4607-f1a9-496d-9516-9de18830abdc None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:52.976825 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a53bd1-fa9d-4e08-8a28-a980656b61d5 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:53.015381 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364b9e9b-656f-405c-ba97-c2755d2bc109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:53.016163 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b9e9b-656f-405c-ba97-c2755d2bc109 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:53.016382 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b9e9b-656f-405c-ba97-c2755d2bc109 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:53.016665 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364b9e9b-656f-405c-ba97-c2755d2bc109 None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:53.017485 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364b9e9b-656f-405c-ba97-c2755d2bc109 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:53.017758 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364b9e9b-656f-405c-ba97-c2755d2bc109 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:53.019225 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364b9e9b-656f-405c-ba97-c2755d2bc109 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:53.035960 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8d4607-f1a9-496d-9516-9de18830abdc None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:53.037709 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 260/547] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 6 16:54:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:53.038555 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364b9e9b-656f-405c-ba97-c2755d2bc109 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:53.071908 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a384dbe-a909-4b66-9a39-d31e8063c4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:53.073038 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a384dbe-a909-4b66-9a39-d31e8063c4cc None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:53.073527 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a384dbe-a909-4b66-9a39-d31e8063c4cc None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:53.074018 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a384dbe-a909-4b66-9a39-d31e8063c4cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:53.084882 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a384dbe-a909-4b66-9a39-d31e8063c4cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:53.093445 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-3a384dbe-a909-4b66-9a39-d31e8063c4cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 16:54:53.093445 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:53.093445 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:53.093445 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:53.093445 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:53.093445 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:53.093445 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:53.093445 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:53.093445 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:53.093445 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:53.093445 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:53.093445 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:53.093445 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:53.093445 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:53.093445 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:53.093445 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:53.094821 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:53.094821 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:53.094821 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:53.094821 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:53.094821 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:53.094821 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:53.094821 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:53.094821 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:53.094821 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:53.094821 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:53.094821 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:53.094821 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:53.094821 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:53.094821 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:53.094821 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:53.094821 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:53.095825 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:53.095825 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:53.095825 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:53.095825 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:53.095825 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:53.095825 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:53.095825 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:53.095825 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:53.095825 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:53.095825 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:53.095825 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 16:54:53.095825 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:53.098166 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 288/548] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 6 16:54:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:53.099168 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a384dbe-a909-4b66-9a39-d31e8063c4cc None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:53.153202 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:53.157689 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364b9e9b-656f-405c-ba97-c2755d2bc109 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:53.161241 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:53.195935 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b18670-b2aa-4dec-b404-1d0cf27f281e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:53.196640 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b18670-b2aa-4dec-b404-1d0cf27f281e None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:53.196929 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b18670-b2aa-4dec-b404-1d0cf27f281e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:53.197209 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b18670-b2aa-4dec-b404-1d0cf27f281e None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:53.197940 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b18670-b2aa-4dec-b404-1d0cf27f281e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:53.198252 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b18670-b2aa-4dec-b404-1d0cf27f281e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:53.199644 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b18670-b2aa-4dec-b404-1d0cf27f281e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:53.205412 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a384dbe-a909-4b66-9a39-d31e8063c4cc None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:53.217307 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 261/549] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:54:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:53.217307 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b18670-b2aa-4dec-b404-1d0cf27f281e None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:53.246983 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d867849-87d9-4ea8-8819-c5360c2577b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:53.248107 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d867849-87d9-4ea8-8819-c5360c2577b6 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:53.248627 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d867849-87d9-4ea8-8819-c5360c2577b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:53.249110 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d867849-87d9-4ea8-8819-c5360c2577b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:53.250145 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d867849-87d9-4ea8-8819-c5360c2577b6 None admin] RBAC: Adding query filter params (role.id=750bca9c32ac4aa49adcbee30463f1fa, scope.project.id=7938b0346d4240ab89774327d09a89ae, user.id=5f479458a68b4daf9f3a7ae53d6c3763) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:53.250684 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d867849-87d9-4ea8-8819-c5360c2577b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:53.252372 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d867849-87d9-4ea8-8819-c5360c2577b6 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:53.258779 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 289/550] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:54:53 2022] GET /identity/v3/role_assignments?user.id=5f479458a68b4daf9f3a7ae53d6c3763&scope.project.id=7938b0346d4240ab89774327d09a89ae&role.id=750bca9c32ac4aa49adcbee30463f1fa => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:53.259842 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d867849-87d9-4ea8-8819-c5360c2577b6 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:53.383008 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b18670-b2aa-4dec-b404-1d0cf27f281e None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:53.389159 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d867849-87d9-4ea8-8819-c5360c2577b6 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:53.436231 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:53.448496 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be65daf6-d2be-4fec-be09-1573099ddd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:53.449508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65daf6-d2be-4fec-be09-1573099ddd46 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:53.449985 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65daf6-d2be-4fec-be09-1573099ddd46 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:53.450447 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be65daf6-d2be-4fec-be09-1573099ddd46 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:53.451422 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be65daf6-d2be-4fec-be09-1573099ddd46 None admin] RBAC: Adding query filter params (name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:53.451957 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be65daf6-d2be-4fec-be09-1573099ddd46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:53.454210 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be65daf6-d2be-4fec-be09-1573099ddd46 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:53.464563 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 262/551] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 6 16:54:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:53.468466 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e13da-a48e-45ae-ab6c-9e958b450d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:53.469544 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e13da-a48e-45ae-ab6c-9e958b450d02 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:53.470706 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be65daf6-d2be-4fec-be09-1573099ddd46 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:53.472034 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e13da-a48e-45ae-ab6c-9e958b450d02 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:53.474423 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e13da-a48e-45ae-ab6c-9e958b450d02 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:53.489387 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0e13da-a48e-45ae-ab6c-9e958b450d02 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:53.492861 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0e13da-a48e-45ae-ab6c-9e958b450d02 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:53.509957 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:53.510604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-3f0e13da-a48e-45ae-ab6c-9e958b450d02 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 06 16:54:53.510604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:53.510604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:53.510604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:53.510604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:53.510604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:53.510604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:53.510604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:53.510604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:53.510604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:53.510604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:53.510604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:53.510604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 16:54:53.510604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 16:54:53.510604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 16:54:53.510604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 16:54:53.511520 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:53.511520 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:53.511520 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 16:54:53.511520 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:54:53.511520 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 16:54:53.511520 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:54:53.511520 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:53.511520 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:53.511520 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:53.511520 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:53.511520 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:53.511520 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:53.511520 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:53.511520 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:53.511520 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:53.511520 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:53.511520 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:53.511520 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:53.512479 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:53.512479 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:53.512479 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 16:54:53.512479 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 16:54:53.512479 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 16:54:53.512479 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 16:54:53.512479 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 16:54:53.512479 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 16:54:53.512479 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 06 16:54:53.512479 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:53.514704 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 290/552] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 6 16:54:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:53.520308 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e13da-a48e-45ae-ab6c-9e958b450d02 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:53.612597 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be65daf6-d2be-4fec-be09-1573099ddd46 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:53.653957 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9789bd3-d025-4d8d-81f6-8fa8431148ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:53.654768 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9789bd3-d025-4d8d-81f6-8fa8431148ba None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:53.655057 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9789bd3-d025-4d8d-81f6-8fa8431148ba None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:53.655336 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9789bd3-d025-4d8d-81f6-8fa8431148ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:53.666119 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9789bd3-d025-4d8d-81f6-8fa8431148ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:53.667713 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9789bd3-d025-4d8d-81f6-8fa8431148ba None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:53.690344 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-f9789bd3-d025-4d8d-81f6-8fa8431148ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:53.690344 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:53.690344 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:53.690344 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:53.690344 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:53.690344 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:53.690344 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:53.690344 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:53.690344 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:53.690344 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:53.690344 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:53.690344 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:53.690344 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:54:53.690344 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:54:53.690344 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:54:53.691212 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:54:53.691212 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:53.691212 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:53.691212 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:53.691212 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:53.691212 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:53.691212 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:53.691212 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:53.691212 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:53.691212 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:53.691212 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:53.691212 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:53.691212 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:53.691212 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:53.691212 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:53.691212 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:53.691212 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:53.692167 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:54:53.692167 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:54:53.692167 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:53.692167 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:53.692167 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 263/553] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:54:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:53.693451 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e405b5-cc5d-4a1e-8d54-2cc18ca8bb9f None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:53.693451 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e405b5-cc5d-4a1e-8d54-2cc18ca8bb9f None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:53.693725 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e405b5-cc5d-4a1e-8d54-2cc18ca8bb9f None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:53.700134 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e13da-a48e-45ae-ab6c-9e958b450d02 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:53.700741 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 264/554] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:53.701596 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e405b5-cc5d-4a1e-8d54-2cc18ca8bb9f None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:53.742977 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2206c74-48ac-4413-ac42-f7d6ec468586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:53.744348 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2206c74-48ac-4413-ac42-f7d6ec468586 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:53.744642 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2206c74-48ac-4413-ac42-f7d6ec468586 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:53.744968 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2206c74-48ac-4413-ac42-f7d6ec468586 None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:53.745819 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2206c74-48ac-4413-ac42-f7d6ec468586 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:53.746133 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2206c74-48ac-4413-ac42-f7d6ec468586 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:53.747748 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2206c74-48ac-4413-ac42-f7d6ec468586 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:53.783511 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:53.795218 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 291/555] 10.222.0.40 () {56 vars in 1135 bytes} [Wed Jul 6 16:54:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:54:53.796655 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845cb905-b49f-457f-bb81-e0935d91ee28 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:53.796953 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845cb905-b49f-457f-bb81-e0935d91ee28 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:53.797235 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845cb905-b49f-457f-bb81-e0935d91ee28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:53.814338 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-845cb905-b49f-457f-bb81-e0935d91ee28 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:53.815778 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-845cb905-b49f-457f-bb81-e0935d91ee28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:53.816556 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e405b5-cc5d-4a1e-8d54-2cc18ca8bb9f None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:53.854131 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450ffc7-6711-4c02-ac2d-1036a7f5e6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:53.854890 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450ffc7-6711-4c02-ac2d-1036a7f5e6fc None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:53.855243 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450ffc7-6711-4c02-ac2d-1036a7f5e6fc None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:53.855677 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450ffc7-6711-4c02-ac2d-1036a7f5e6fc None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:53.856546 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:53.857538 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d450ffc7-6711-4c02-ac2d-1036a7f5e6fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:53.857852 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d450ffc7-6711-4c02-ac2d-1036a7f5e6fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:53.859230 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d450ffc7-6711-4c02-ac2d-1036a7f5e6fc None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:53.872036 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 265/556] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:54:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:53.872900 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450ffc7-6711-4c02-ac2d-1036a7f5e6fc None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:53.898171 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 292/557] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:53.900004 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845cb905-b49f-457f-bb81-e0935d91ee28 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:54.020278 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845cb905-b49f-457f-bb81-e0935d91ee28 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:54.026706 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450ffc7-6711-4c02-ac2d-1036a7f5e6fc None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:54.060858 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:54.065985 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c3b188-8ed6-4fc3-a3eb-27499526ae8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:54.066769 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3b188-8ed6-4fc3-a3eb-27499526ae8f None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:54.067100 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3b188-8ed6-4fc3-a3eb-27499526ae8f None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:54.067381 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c3b188-8ed6-4fc3-a3eb-27499526ae8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:54.070582 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56df195-d7f7-4c77-88aa-d0cd5ffc0627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:54.072189 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56df195-d7f7-4c77-88aa-d0cd5ffc0627 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:54.072475 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56df195-d7f7-4c77-88aa-d0cd5ffc0627 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:54.072756 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56df195-d7f7-4c77-88aa-d0cd5ffc0627 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:54.080726 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56df195-d7f7-4c77-88aa-d0cd5ffc0627 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:54.082287 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56df195-d7f7-4c77-88aa-d0cd5ffc0627 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:54.098470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-b56df195-d7f7-4c77-88aa-d0cd5ffc0627 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 06 16:54:54.098470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:54.098470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:54.098470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:54.098470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:54.098470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:54.098470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:54.098470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:54.098470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:54.098470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:54.098470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:54.098470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:54.098470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 16:54:54.098470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 16:54:54.098470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 16:54:54.098470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 16:54:54.100150 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:54.100150 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:54.100150 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 16:54:54.100150 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:54:54.100150 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 16:54:54.100150 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:54:54.100150 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:54.100150 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:54.100150 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:54.100150 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:54.100150 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:54.100150 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:54.100150 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:54.100150 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:54.100150 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:54.100150 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:54.100150 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:54.100150 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:54.101298 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:54.101298 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:54.101298 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 16:54:54.101298 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 16:54:54.101298 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 16:54:54.101298 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 16:54:54.101298 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 16:54:54.101298 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 16:54:54.101298 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 06 16:54:54.101298 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:54.106290 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 293/558] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Jul 6 16:54:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:54.107212 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56df195-d7f7-4c77-88aa-d0cd5ffc0627 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:54.140167 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:54.158556 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c3b188-8ed6-4fc3-a3eb-27499526ae8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:54.160544 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c3b188-8ed6-4fc3-a3eb-27499526ae8f None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:54.213303 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56df195-d7f7-4c77-88aa-d0cd5ffc0627 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:54.259928 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe7f71-cb50-4b88-9f13-281defa1ddde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:54.261226 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe7f71-cb50-4b88-9f13-281defa1ddde None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:54.262034 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe7f71-cb50-4b88-9f13-281defa1ddde None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:54.262512 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe7f71-cb50-4b88-9f13-281defa1ddde None admin] PATH_INFO: `/v3/roles/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:54.277871 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbe7f71-cb50-4b88-9f13-281defa1ddde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:54.279217 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 266/559] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:54:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 16:54:54.280272 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c3b188-8ed6-4fc3-a3eb-27499526ae8f None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:54.287602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-2cbe7f71-cb50-4b88-9f13-281defa1ddde None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 16:54:54.287602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:54.287602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:54.287602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:54.287602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:54.287602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:54.287602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:54.287602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:54.287602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:54.287602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:54.287602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:54.287602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:54.287602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:54.287602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:54.287602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:54.287602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:54.288702 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:54.288702 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:54.288702 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:54.288702 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:54.288702 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:54.288702 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:54.288702 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:54.288702 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:54.288702 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:54.288702 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:54.288702 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:54.288702 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:54.288702 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:54.288702 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:54.288702 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:54.288702 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:54.289521 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:54.289521 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:54.289521 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:54.289521 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:54.289521 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:54.289521 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:54.289521 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:54.289521 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:54.289521 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:54.289521 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:54.289521 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 16:54:54.289521 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:54.293786 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 294/560] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 6 16:54:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:54.294539 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe7f71-cb50-4b88-9f13-281defa1ddde None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:54.322269 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:54.394669 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:54.395187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c3b188-8ed6-4fc3-a3eb-27499526ae8f None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:54.404043 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe7f71-cb50-4b88-9f13-281defa1ddde None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:54.433858 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4bd448-10c0-4a5c-9a80-db3b9a923258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:54.434536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4bd448-10c0-4a5c-9a80-db3b9a923258 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:54.434822 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4bd448-10c0-4a5c-9a80-db3b9a923258 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:54.435097 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4bd448-10c0-4a5c-9a80-db3b9a923258 None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:54.435957 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4bd448-10c0-4a5c-9a80-db3b9a923258 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:54.436447 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4bd448-10c0-4a5c-9a80-db3b9a923258 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:54.437774 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4bd448-10c0-4a5c-9a80-db3b9a923258 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:54.441030 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0164a3-92de-43eb-939c-5826fab6df7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:54.441821 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0164a3-92de-43eb-939c-5826fab6df7c None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:54.442107 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0164a3-92de-43eb-939c-5826fab6df7c None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:54.442387 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0164a3-92de-43eb-939c-5826fab6df7c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:54.452887 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0164a3-92de-43eb-939c-5826fab6df7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:54.454316 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0164a3-92de-43eb-939c-5826fab6df7c None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:54.460779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 267/561] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Jul 6 16:54:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:54.461856 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4bd448-10c0-4a5c-9a80-db3b9a923258 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:54.472506 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-eb0164a3-92de-43eb-939c-5826fab6df7c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:54.472506 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:54.472506 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:54.472506 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:54.472506 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:54.472506 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:54.472506 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:54.472506 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:54.472506 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:54.472506 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:54.472506 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:54.472506 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:54.472506 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:54:54.472506 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:54:54.472506 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:54:54.473439 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:54:54.473439 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:54.473439 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:54.473439 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:54.473439 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:54.473439 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:54.473439 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:54.473439 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:54.473439 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:54.473439 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:54.473439 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:54.473439 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:54.473439 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:54.473439 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:54.473439 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:54.473439 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:54.473439 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:54.474304 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:54:54.474304 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:54:54.474304 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:54.474304 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:54.474304 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 295/562] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:54:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:54.474948 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5f55b-13ae-4d3a-8570-21ec097480a2 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:54.475241 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5f55b-13ae-4d3a-8570-21ec097480a2 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:54.475520 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5f55b-13ae-4d3a-8570-21ec097480a2 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:54.476944 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 296/563] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:54.478016 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda5f55b-13ae-4d3a-8570-21ec097480a2 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:54.564081 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:54.575268 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4bd448-10c0-4a5c-9a80-db3b9a923258 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:54.587400 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda5f55b-13ae-4d3a-8570-21ec097480a2 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:54.612524 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbd7975-77db-4298-a4b4-656f7070bedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:54.613198 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd7975-77db-4298-a4b4-656f7070bedc None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:54.613483 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd7975-77db-4298-a4b4-656f7070bedc None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:54.613761 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbd7975-77db-4298-a4b4-656f7070bedc None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:54.614504 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbd7975-77db-4298-a4b4-656f7070bedc None admin] RBAC: Adding query filter params (name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:54.614810 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbd7975-77db-4298-a4b4-656f7070bedc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:54.616221 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbd7975-77db-4298-a4b4-656f7070bedc None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:54.622933 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1f2dcf-3529-4c50-a718-e92426c3e0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:54.623714 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f2dcf-3529-4c50-a718-e92426c3e0a8 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:54.624063 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f2dcf-3529-4c50-a718-e92426c3e0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:54.624283 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1f2dcf-3529-4c50-a718-e92426c3e0a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:54.629669 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 268/564] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 6 16:54:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:54.630406 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbd7975-77db-4298-a4b4-656f7070bedc None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:54.638326 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1f2dcf-3529-4c50-a718-e92426c3e0a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:54.639846 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1f2dcf-3529-4c50-a718-e92426c3e0a8 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:54.643395 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:54.659867 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-7f1f2dcf-3529-4c50-a718-e92426c3e0a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:54.659867 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:54.659867 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:54.659867 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:54.659867 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:54.659867 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:54.659867 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:54.659867 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:54.659867 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:54.659867 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:54.659867 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:54.659867 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:54.659867 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:54:54.659867 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:54:54.659867 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:54:54.666609 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:54:54.666609 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:54.666609 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:54.666609 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:54.666609 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:54.666609 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:54.666609 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:54.666609 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:54.666609 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:54.666609 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:54.666609 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:54.666609 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:54.666609 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:54.666609 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:54.666609 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:54.666609 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:54.666609 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:54.667964 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:54:54.667964 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:54:54.667964 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:54.667964 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:54.667964 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 297/565] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:54:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:54.667964 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd52e5e3-60a2-4cff-bba9-498d36fb5dbf None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:54.667964 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd52e5e3-60a2-4cff-bba9-498d36fb5dbf None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:54.667964 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd52e5e3-60a2-4cff-bba9-498d36fb5dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:54.680816 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-bd52e5e3-60a2-4cff-bba9-498d36fb5dbf None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:54.683106 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-bd52e5e3-60a2-4cff-bba9-498d36fb5dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:54.748947 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbd7975-77db-4298-a4b4-656f7070bedc None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:54.758006 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 298/566] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:54.758946 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd52e5e3-60a2-4cff-bba9-498d36fb5dbf None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:54.791904 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5169f-43d0-4afa-8101-d9864244f533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:54.792589 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5169f-43d0-4afa-8101-d9864244f533 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:54.792878 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5169f-43d0-4afa-8101-d9864244f533 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:54.793157 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a5169f-43d0-4afa-8101-d9864244f533 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:54.793890 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a5169f-43d0-4afa-8101-d9864244f533 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:54.794187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a5169f-43d0-4afa-8101-d9864244f533 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:54.795575 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a5169f-43d0-4afa-8101-d9864244f533 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:54.807947 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 269/567] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:54:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:54.808741 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5169f-43d0-4afa-8101-d9864244f533 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:54.823958 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:54.881355 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd52e5e3-60a2-4cff-bba9-498d36fb5dbf None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:54.908891 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a5169f-43d0-4afa-8101-d9864244f533 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:54.909071 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:54.918219 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc288e26-de97-4acf-99a3-0bbb4d0266ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:54.927577 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc288e26-de97-4acf-99a3-0bbb4d0266ef None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:54.927577 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc288e26-de97-4acf-99a3-0bbb4d0266ef None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:54.927577 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc288e26-de97-4acf-99a3-0bbb4d0266ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:54.930285 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc288e26-de97-4acf-99a3-0bbb4d0266ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:54.931786 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc288e26-de97-4acf-99a3-0bbb4d0266ef None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:54.943094 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5fb3b5-e29b-4fc3-8e9b-57581a4ce03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:54.943781 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5fb3b5-e29b-4fc3-8e9b-57581a4ce03d None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:54.944071 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5fb3b5-e29b-4fc3-8e9b-57581a4ce03d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:54.944346 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5fb3b5-e29b-4fc3-8e9b-57581a4ce03d None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:54.945067 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5fb3b5-e29b-4fc3-8e9b-57581a4ce03d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:54.945377 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5fb3b5-e29b-4fc3-8e9b-57581a4ce03d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:54.947270 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5fb3b5-e29b-4fc3-8e9b-57581a4ce03d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:54.950402 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-cc288e26-de97-4acf-99a3-0bbb4d0266ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:54.950402 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:54.950402 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:54.950402 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:54.950402 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:54.950402 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:54.950402 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:54.950402 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:54.950402 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:54.950402 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:54.950402 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:54.950402 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:54.950402 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:54:54.950402 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:54:54.950402 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:54:54.951232 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:54:54.951232 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:54.951232 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:54.951232 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:54.951232 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:54.951232 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:54.951232 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:54.951232 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:54.951232 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:54.951232 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:54.951232 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:54.951232 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:54.951232 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:54.951232 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:54.951232 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:54.951232 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:54.951232 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:54.952527 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:54:54.952527 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:54:54.952527 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:54.952527 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:54.956321 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 299/568] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:54:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:54.957150 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc288e26-de97-4acf-99a3-0bbb4d0266ef None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:54.963986 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 270/569] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:54:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:54.963986 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5fb3b5-e29b-4fc3-8e9b-57581a4ce03d None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:55.040237 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:55.064436 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc288e26-de97-4acf-99a3-0bbb4d0266ef None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:55.074532 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5fb3b5-e29b-4fc3-8e9b-57581a4ce03d None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:55.106516 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7729391-6b62-48e7-a795-226ce75a1a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:55.107511 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7729391-6b62-48e7-a795-226ce75a1a4b None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:55.107907 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7729391-6b62-48e7-a795-226ce75a1a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:55.108187 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7729391-6b62-48e7-a795-226ce75a1a4b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:55.110200 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c426225-49e2-4c1f-8295-a5113a4e2a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:55.111038 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c426225-49e2-4c1f-8295-a5113a4e2a4a None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:55.111319 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c426225-49e2-4c1f-8295-a5113a4e2a4a None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:55.111603 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c426225-49e2-4c1f-8295-a5113a4e2a4a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:55.118588 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c426225-49e2-4c1f-8295-a5113a4e2a4a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:55.120063 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c426225-49e2-4c1f-8295-a5113a4e2a4a None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:55.120871 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7729391-6b62-48e7-a795-226ce75a1a4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:55.129498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-e7729391-6b62-48e7-a795-226ce75a1a4b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 16:54:55.129498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:55.129498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:55.129498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:55.129498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:55.129498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:55.129498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:55.129498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:55.129498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:55.129498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:55.129498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:55.129498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:55.129498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:55.129498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:55.129498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:55.129498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:55.130380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:55.130380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:55.130380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:55.130380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:55.130380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:55.130380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:55.130380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:55.130380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:55.130380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:55.130380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:55.130380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:55.130380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:55.130380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:55.130380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:55.130380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:55.130380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:55.131262 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:55.131262 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:55.131262 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:55.131262 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:55.131262 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:55.131262 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:55.131262 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:55.131262 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:55.131262 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:55.131262 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:55.131262 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 16:54:55.131262 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:55.132421 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 300/570] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 6 16:54:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:55.133189 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7729391-6b62-48e7-a795-226ce75a1a4b None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:55.134886 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-5c426225-49e2-4c1f-8295-a5113a4e2a4a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:54:55.134886 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:55.134886 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:55.134886 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:55.134886 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:55.134886 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:55.134886 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:55.134886 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:55.134886 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:55.134886 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:55.134886 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:55.134886 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:55.134886 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 16:54:55.134886 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 16:54:55.134886 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 16:54:55.135722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 16:54:55.135722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:55.135722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:55.135722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:55.135722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:55.135722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:55.135722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:55.135722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:55.135722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:55.135722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:55.135722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:55.135722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:55.135722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:55.135722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:55.135722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:55.135722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:55.135722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:55.136718 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 16:54:55.136718 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 16:54:55.136718 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 16:54:55.136718 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 16:54:55.136718 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 16:54:55.136718 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 16:54:55.136718 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:54:55.136718 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:55.140639 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 271/571] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:54:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:55.141473 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c426225-49e2-4c1f-8295-a5113a4e2a4a None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:55.184405 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:55.247031 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c426225-49e2-4c1f-8295-a5113a4e2a4a None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:55.250821 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7729391-6b62-48e7-a795-226ce75a1a4b None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:55.277308 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:55.282430 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f354c0ae-07e8-431c-ad7f-b7c68a6aa794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:55.283279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354c0ae-07e8-431c-ad7f-b7c68a6aa794 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:55.283566 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354c0ae-07e8-431c-ad7f-b7c68a6aa794 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:55.283987 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354c0ae-07e8-431c-ad7f-b7c68a6aa794 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:55.288594 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c374602d-9f95-4e3f-b08b-f59d80a57e0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:55.289262 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374602d-9f95-4e3f-b08b-f59d80a57e0b None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:55.289545 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374602d-9f95-4e3f-b08b-f59d80a57e0b None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:55.289865 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c374602d-9f95-4e3f-b08b-f59d80a57e0b None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:55.290586 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c374602d-9f95-4e3f-b08b-f59d80a57e0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:55.290897 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c374602d-9f95-4e3f-b08b-f59d80a57e0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:55.291139 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f354c0ae-07e8-431c-ad7f-b7c68a6aa794 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:55.292297 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c374602d-9f95-4e3f-b08b-f59d80a57e0b None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:55.292672 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f354c0ae-07e8-431c-ad7f-b7c68a6aa794 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:55.304311 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 301/572] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:54:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:55.305183 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c374602d-9f95-4e3f-b08b-f59d80a57e0b None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:55.308655 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-f354c0ae-07e8-431c-ad7f-b7c68a6aa794 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:54:55.308655 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:55.308655 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:55.308655 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:55.308655 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:55.308655 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:55.308655 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:55.308655 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:55.308655 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:55.308655 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:55.308655 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:55.308655 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:55.308655 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 16:54:55.308655 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 16:54:55.308655 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 16:54:55.309530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 16:54:55.309530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:55.309530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:55.309530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:55.309530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:55.309530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:55.309530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:55.309530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:55.309530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:55.309530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:55.309530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:55.309530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:55.309530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:55.309530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:55.309530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:55.309530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:55.309530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:55.310431 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 16:54:55.310431 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 16:54:55.310431 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 16:54:55.310431 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 16:54:55.310431 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 16:54:55.310431 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 16:54:55.310431 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:54:55.310431 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:55.312262 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 272/573] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:54:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:55.313209 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f354c0ae-07e8-431c-ad7f-b7c68a6aa794 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:55.410536 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:55.415917 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c374602d-9f95-4e3f-b08b-f59d80a57e0b None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:55.429293 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f354c0ae-07e8-431c-ad7f-b7c68a6aa794 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:55.451328 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9be0338-a7f6-43d3-8e3e-571f12a1f362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:55.452041 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9be0338-a7f6-43d3-8e3e-571f12a1f362 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:55.452325 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9be0338-a7f6-43d3-8e3e-571f12a1f362 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:55.452600 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9be0338-a7f6-43d3-8e3e-571f12a1f362 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:55.453342 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9be0338-a7f6-43d3-8e3e-571f12a1f362 None admin] RBAC: Adding query filter params (name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:55.453641 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9be0338-a7f6-43d3-8e3e-571f12a1f362 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:55.455004 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9be0338-a7f6-43d3-8e3e-571f12a1f362 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:55.464122 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 302/574] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 6 16:54:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:55.464912 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9be0338-a7f6-43d3-8e3e-571f12a1f362 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:55.470370 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb5016-af9b-4244-b751-b735906c2894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:55.471045 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb5016-af9b-4244-b751-b735906c2894 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:55.471328 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb5016-af9b-4244-b751-b735906c2894 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:55.471602 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cb5016-af9b-4244-b751-b735906c2894 None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:55.472417 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb5016-af9b-4244-b751-b735906c2894 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:55.472726 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb5016-af9b-4244-b751-b735906c2894 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:55.474156 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cb5016-af9b-4244-b751-b735906c2894 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:55.491808 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 273/575] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 6 16:54:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:55.491808 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb5016-af9b-4244-b751-b735906c2894 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:55.543709 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:55.571084 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9be0338-a7f6-43d3-8e3e-571f12a1f362 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:55.590736 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cb5016-af9b-4244-b751-b735906c2894 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:55.605669 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef166f1-8044-4c58-a959-e46d406ae0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:55.606463 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef166f1-8044-4c58-a959-e46d406ae0d0 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:55.606744 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef166f1-8044-4c58-a959-e46d406ae0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:55.607018 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef166f1-8044-4c58-a959-e46d406ae0d0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:55.614426 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef166f1-8044-4c58-a959-e46d406ae0d0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:55.615873 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef166f1-8044-4c58-a959-e46d406ae0d0 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:55.625044 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094b693-2158-4f2b-bc84-2c961bb8b6e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:55.625700 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094b693-2158-4f2b-bc84-2c961bb8b6e5 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:55.625984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094b693-2158-4f2b-bc84-2c961bb8b6e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:55.626261 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5094b693-2158-4f2b-bc84-2c961bb8b6e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:55.627042 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5094b693-2158-4f2b-bc84-2c961bb8b6e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:55.627349 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5094b693-2158-4f2b-bc84-2c961bb8b6e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:55.628792 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5094b693-2158-4f2b-bc84-2c961bb8b6e5 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:55.629982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-5ef166f1-8044-4c58-a959-e46d406ae0d0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 06 16:54:55.629982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:55.629982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:55.629982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:55.629982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:55.629982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:55.629982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:55.629982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:55.629982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:55.629982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:55.629982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:55.629982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:55.629982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 16:54:55.629982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 16:54:55.629982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 16:54:55.629982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 16:54:55.630925 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:55.630925 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:55.630925 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 16:54:55.630925 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:54:55.630925 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 16:54:55.630925 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:54:55.630925 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:55.630925 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:55.630925 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:55.630925 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:55.630925 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:55.630925 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:55.630925 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:55.630925 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:55.630925 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:55.630925 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:55.630925 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:55.630925 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:55.631926 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:55.631926 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:55.631926 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 16:54:55.631926 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 16:54:55.631926 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 16:54:55.631926 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 16:54:55.631926 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 16:54:55.631926 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 16:54:55.631926 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 06 16:54:55.631926 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:55.631926 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 303/576] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:54:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:55.632511 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef166f1-8044-4c58-a959-e46d406ae0d0 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:55.641958 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 274/577] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 6 16:54:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:55.642766 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094b693-2158-4f2b-bc84-2c961bb8b6e5 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:55.654684 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:55.740760 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef166f1-8044-4c58-a959-e46d406ae0d0 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:55.742061 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5094b693-2158-4f2b-bc84-2c961bb8b6e5 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:55.755212 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:55.778908 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cc9fef-5bb1-4686-826c-53c1127f02a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:55.779150 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0998b572-0e2f-441b-8003-6b3d941c0142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:55.779706 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc9fef-5bb1-4686-826c-53c1127f02a5 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:55.779839 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998b572-0e2f-441b-8003-6b3d941c0142 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:55.780019 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc9fef-5bb1-4686-826c-53c1127f02a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:55.780111 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998b572-0e2f-441b-8003-6b3d941c0142 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:55.780286 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cc9fef-5bb1-4686-826c-53c1127f02a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:55.780379 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0998b572-0e2f-441b-8003-6b3d941c0142 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:55.781204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0998b572-0e2f-441b-8003-6b3d941c0142 None admin] RBAC: Adding query filter params (role.id=750bca9c32ac4aa49adcbee30463f1fa, scope.project.id=7938b0346d4240ab89774327d09a89ae, user.id=75acacd30e9a4cf2ad58b072b49e3f05) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:55.781519 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0998b572-0e2f-441b-8003-6b3d941c0142 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:55.783225 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0998b572-0e2f-441b-8003-6b3d941c0142 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:55.789134 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 275/578] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:54:55 2022] GET /identity/v3/role_assignments?user.id=75acacd30e9a4cf2ad58b072b49e3f05&scope.project.id=7938b0346d4240ab89774327d09a89ae&role.id=750bca9c32ac4aa49adcbee30463f1fa => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:55.790220 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412f86e-2171-4813-ad0d-c8e15077a165 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:55.790329 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cc9fef-5bb1-4686-826c-53c1127f02a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:55.790512 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412f86e-2171-4813-ad0d-c8e15077a165 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:55.790791 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9412f86e-2171-4813-ad0d-c8e15077a165 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:55.791781 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cc9fef-5bb1-4686-826c-53c1127f02a5 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:55.795907 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 276/579] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:55.795907 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9412f86e-2171-4813-ad0d-c8e15077a165 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:55.816010 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-41cc9fef-5bb1-4686-826c-53c1127f02a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:55.816010 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:55.816010 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:55.816010 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:55.816010 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:55.816010 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:55.816010 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:55.816010 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:55.816010 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:55.816010 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:55.816010 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:55.816010 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:55.816010 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:54:55.816010 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:54:55.816010 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:54:55.817169 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:54:55.817169 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:55.817169 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:55.817169 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:55.817169 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:55.817169 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:55.817169 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:55.817169 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:55.817169 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:55.817169 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:55.817169 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:55.817169 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:55.817169 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:55.817169 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:55.817169 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:55.817169 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:55.817169 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:55.818065 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:54:55.818065 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:54:55.818065 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:55.818065 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:55.818065 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 304/580] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:54:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:55.818065 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cc9fef-5bb1-4686-826c-53c1127f02a5 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:55.900633 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:55.915784 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9412f86e-2171-4813-ad0d-c8e15077a165 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:55.944203 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cc9fef-5bb1-4686-826c-53c1127f02a5 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:55.989826 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d84e34-421a-45a5-a4c0-577aeab169a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:55.990593 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d84e34-421a-45a5-a4c0-577aeab169a0 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:55.990984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d84e34-421a-45a5-a4c0-577aeab169a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:55.991376 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d84e34-421a-45a5-a4c0-577aeab169a0 None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:55.992361 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d84e34-421a-45a5-a4c0-577aeab169a0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:55.993304 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d84e34-421a-45a5-a4c0-577aeab169a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:55.994424 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e2f509-798d-4d5f-94db-a77dcc83cac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:55.994949 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d84e34-421a-45a5-a4c0-577aeab169a0 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:55.995312 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2f509-798d-4d5f-94db-a77dcc83cac4 None admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:55.995597 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2f509-798d-4d5f-94db-a77dcc83cac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:55.995899 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e2f509-798d-4d5f-94db-a77dcc83cac4 None admin] PATH_INFO: `/v3/projects/7938b0346d4240ab89774327d09a89ae/users/af26ef96dc9d47dc91549b736939268e/roles/750bca9c32ac4aa49adcbee30463f1fa` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:56.011880 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e2f509-798d-4d5f-94db-a77dcc83cac4 None admin] RBAC: Authorizing `identity:create_grant(project_id=7938b0346d4240ab89774327d09a89ae, user_id=af26ef96dc9d47dc91549b736939268e, role_id=750bca9c32ac4aa49adcbee30463f1fa)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:56.013319 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e2f509-798d-4d5f-94db-a77dcc83cac4 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:56.018751 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 277/581] 10.222.0.40 () {56 vars in 1137 bytes} [Wed Jul 6 16:54:55 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:56.020176 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255e013-9e5c-48cd-9cd5-881b2f4b41d0 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:56.020543 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255e013-9e5c-48cd-9cd5-881b2f4b41d0 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:56.020901 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7255e013-9e5c-48cd-9cd5-881b2f4b41d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:56.041748 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-7255e013-9e5c-48cd-9cd5-881b2f4b41d0 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:56.043623 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-7255e013-9e5c-48cd-9cd5-881b2f4b41d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:56.073015 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:56.104586 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 305/582] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 6 16:54:55 2022] PUT /identity/v3/projects/7938b0346d4240ab89774327d09a89ae/users/af26ef96dc9d47dc91549b736939268e/roles/750bca9c32ac4aa49adcbee30463f1fa => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 16:54:56.105825 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e2f509-798d-4d5f-94db-a77dcc83cac4 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:56.126284 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 278/583] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:56.127455 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7255e013-9e5c-48cd-9cd5-881b2f4b41d0 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:56.215397 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e2f509-798d-4d5f-94db-a77dcc83cac4 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:56.241942 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:56.249013 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7255e013-9e5c-48cd-9cd5-881b2f4b41d0 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:56.257351 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2d4d4c-ad90-4800-9e6e-d1f9d94911f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:56.258251 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d4d4c-ad90-4800-9e6e-d1f9d94911f1 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:56.258635 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d4d4c-ad90-4800-9e6e-d1f9d94911f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:56.259000 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2d4d4c-ad90-4800-9e6e-d1f9d94911f1 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:56.259948 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2d4d4c-ad90-4800-9e6e-d1f9d94911f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:56.260325 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2d4d4c-ad90-4800-9e6e-d1f9d94911f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:56.261929 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2d4d4c-ad90-4800-9e6e-d1f9d94911f1 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:56.281202 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 306/584] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:54:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:56.282803 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2d4d4c-ad90-4800-9e6e-d1f9d94911f1 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:56.289532 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d70520-9840-4ff6-aa20-65035fd4e89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:56.290918 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d70520-9840-4ff6-aa20-65035fd4e89d None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:56.291204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d70520-9840-4ff6-aa20-65035fd4e89d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:56.291483 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d70520-9840-4ff6-aa20-65035fd4e89d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:56.303217 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d70520-9840-4ff6-aa20-65035fd4e89d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:56.304770 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d70520-9840-4ff6-aa20-65035fd4e89d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:56.327489 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-28d70520-9840-4ff6-aa20-65035fd4e89d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:56.327489 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:56.327489 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:56.327489 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:56.327489 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:56.327489 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:56.327489 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:56.327489 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:56.327489 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:56.327489 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:56.327489 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:56.327489 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:56.327489 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:54:56.327489 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:54:56.327489 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:54:56.328505 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:54:56.328505 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:56.328505 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:56.328505 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:56.328505 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:56.328505 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:56.328505 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:56.328505 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:56.328505 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:56.328505 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:56.328505 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:56.328505 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:56.328505 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:56.328505 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:56.328505 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:56.328505 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:56.328505 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:56.329378 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:54:56.329378 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:54:56.329378 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:56.329378 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:56.333600 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 279/585] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:54:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:56.334544 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d70520-9840-4ff6-aa20-65035fd4e89d None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:56.367783 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:56.457422 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2d4d4c-ad90-4800-9e6e-d1f9d94911f1 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:56.482753 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d70520-9840-4ff6-aa20-65035fd4e89d None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:56.497268 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1385665-c35d-4b18-b50e-81a5ef8ab23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:56.498481 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1385665-c35d-4b18-b50e-81a5ef8ab23b None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:56.498772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1385665-c35d-4b18-b50e-81a5ef8ab23b None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:56.499194 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1385665-c35d-4b18-b50e-81a5ef8ab23b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:56.515134 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1385665-c35d-4b18-b50e-81a5ef8ab23b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:56.519359 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8898298-b880-41e4-8ff0-2eeadf3d7b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:56.520270 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8898298-b880-41e4-8ff0-2eeadf3d7b19 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:56.520624 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8898298-b880-41e4-8ff0-2eeadf3d7b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:56.520969 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8898298-b880-41e4-8ff0-2eeadf3d7b19 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:56.524604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-f1385665-c35d-4b18-b50e-81a5ef8ab23b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 16:54:56.524604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:56.524604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:56.524604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:56.524604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:56.524604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:56.524604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:56.524604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:56.524604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:56.524604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:56.524604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:56.524604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:56.524604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:56.524604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:56.524604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:56.524604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:56.526497 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:56.526497 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:56.526497 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:56.526497 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:56.526497 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:56.526497 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:56.526497 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:56.526497 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:56.526497 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:56.526497 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:56.526497 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:56.526497 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:56.526497 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:56.526497 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:56.526497 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:56.526497 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:56.527863 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:56.527863 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:56.527863 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:56.527863 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:56.527863 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:56.527863 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:56.527863 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:56.527863 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:56.527863 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:56.527863 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:56.527863 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 16:54:56.527863 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:56.527863 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 307/586] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 6 16:54:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:56.527863 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1385665-c35d-4b18-b50e-81a5ef8ab23b None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:56.534637 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8898298-b880-41e4-8ff0-2eeadf3d7b19 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:56.536236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8898298-b880-41e4-8ff0-2eeadf3d7b19 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:56.553551 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-a8898298-b880-41e4-8ff0-2eeadf3d7b19 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 06 16:54:56.553551 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:56.553551 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:56.553551 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:56.553551 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:56.553551 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:56.553551 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:56.553551 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:56.553551 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:56.553551 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:56.553551 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:56.553551 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:56.553551 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 16:54:56.553551 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 16:54:56.553551 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 16:54:56.553551 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 16:54:56.554745 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:56.554745 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:56.554745 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 16:54:56.554745 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:54:56.554745 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 16:54:56.554745 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:54:56.554745 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:56.554745 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:56.554745 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:56.554745 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:56.554745 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:56.554745 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:56.554745 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:56.554745 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:56.554745 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:56.554745 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:56.554745 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:56.554745 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:56.556488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:56.556488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:56.556488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 16:54:56.556488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 16:54:56.556488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 16:54:56.556488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 16:54:56.556488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 16:54:56.556488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 16:54:56.556488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 06 16:54:56.556488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:56.558223 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 280/587] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:54:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:56.561617 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8898298-b880-41e4-8ff0-2eeadf3d7b19 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:56.587994 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:56.650906 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:56.656735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1385665-c35d-4b18-b50e-81a5ef8ab23b None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:56.674674 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8898298-b880-41e4-8ff0-2eeadf3d7b19 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:56.695494 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89432332-ae86-449c-af80-9b7ea05ffabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:56.696364 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89432332-ae86-449c-af80-9b7ea05ffabf None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:56.696757 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89432332-ae86-449c-af80-9b7ea05ffabf None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:56.697043 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89432332-ae86-449c-af80-9b7ea05ffabf None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:56.697783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89432332-ae86-449c-af80-9b7ea05ffabf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:56.698096 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89432332-ae86-449c-af80-9b7ea05ffabf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:56.699576 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89432332-ae86-449c-af80-9b7ea05ffabf None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:56.711015 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0adc8-0dc3-4108-ad42-f43eac4628cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:56.711800 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0adc8-0dc3-4108-ad42-f43eac4628cf None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:56.712093 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0adc8-0dc3-4108-ad42-f43eac4628cf None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:56.712376 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb0adc8-0dc3-4108-ad42-f43eac4628cf None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:56.713183 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb0adc8-0dc3-4108-ad42-f43eac4628cf None admin] RBAC: Adding query filter params (name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:56.713507 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb0adc8-0dc3-4108-ad42-f43eac4628cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:56.713686 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 308/588] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:54:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:56.714359 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89432332-ae86-449c-af80-9b7ea05ffabf None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:56.714870 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb0adc8-0dc3-4108-ad42-f43eac4628cf None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:56.741141 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 281/589] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 6 16:54:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:56.742052 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0adc8-0dc3-4108-ad42-f43eac4628cf None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:56.848488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb0adc8-0dc3-4108-ad42-f43eac4628cf None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:56.855600 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89432332-ae86-449c-af80-9b7ea05ffabf None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:56.857812 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:56.885702 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b082f2c5-139a-49a2-b00f-b217098f38ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:56.886510 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b082f2c5-139a-49a2-b00f-b217098f38ee None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:56.886798 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b082f2c5-139a-49a2-b00f-b217098f38ee None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:56.887077 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b082f2c5-139a-49a2-b00f-b217098f38ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:56.894344 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b082f2c5-139a-49a2-b00f-b217098f38ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:56.895937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b082f2c5-139a-49a2-b00f-b217098f38ee None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:56.897957 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:56.911757 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4191098e-80dd-4b4a-b99d-55bec1871997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:56.911757 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191098e-80dd-4b4a-b99d-55bec1871997 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:56.911757 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191098e-80dd-4b4a-b99d-55bec1871997 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:56.911757 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191098e-80dd-4b4a-b99d-55bec1871997 None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:56.912445 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4191098e-80dd-4b4a-b99d-55bec1871997 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:56.912445 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4191098e-80dd-4b4a-b99d-55bec1871997 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:56.912756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-b082f2c5-139a-49a2-b00f-b217098f38ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:54:56.912756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:56.912756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:56.912756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:56.912756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:56.912756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:56.912756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:56.912756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:56.912756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:56.912756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:56.912756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:56.912756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:56.912756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 16:54:56.912756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 16:54:56.912756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 16:54:56.913724 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 16:54:56.913724 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:56.913724 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:56.913724 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:56.913724 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:56.913724 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:56.913724 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:56.913724 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:56.913724 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:56.913724 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:56.913724 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:56.913724 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:56.913724 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:56.913724 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:56.913724 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:56.913724 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:56.913724 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:56.915380 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 16:54:56.915380 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 16:54:56.915380 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 16:54:56.915380 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 16:54:56.915380 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 16:54:56.915380 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 16:54:56.915380 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:54:56.915380 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:56.915853 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4191098e-80dd-4b4a-b99d-55bec1871997 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:56.915937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 282/590] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:54:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:56.915937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b082f2c5-139a-49a2-b00f-b217098f38ee None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:56.941081 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 309/591] 10.222.0.40 () {56 vars in 1137 bytes} [Wed Jul 6 16:54:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:56.942735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4191098e-80dd-4b4a-b99d-55bec1871997 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:57.057568 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b082f2c5-139a-49a2-b00f-b217098f38ee None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:57.074892 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4191098e-80dd-4b4a-b99d-55bec1871997 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:57.105151 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebed452-d9a3-4c47-a774-c63a694110c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:57.106893 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebed452-d9a3-4c47-a774-c63a694110c2 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:57.107631 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebed452-d9a3-4c47-a774-c63a694110c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:57.108137 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebed452-d9a3-4c47-a774-c63a694110c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:57.111245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d3e21-b526-4bef-ba16-cb9559af2a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:57.111941 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d3e21-b526-4bef-ba16-cb9559af2a67 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:57.112228 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d3e21-b526-4bef-ba16-cb9559af2a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:57.112505 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d3e21-b526-4bef-ba16-cb9559af2a67 None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:57.113290 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302d3e21-b526-4bef-ba16-cb9559af2a67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:57.113600 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302d3e21-b526-4bef-ba16-cb9559af2a67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:57.115023 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302d3e21-b526-4bef-ba16-cb9559af2a67 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:57.120883 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebed452-d9a3-4c47-a774-c63a694110c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:57.123157 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebed452-d9a3-4c47-a774-c63a694110c2 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:57.132631 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 310/592] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 6 16:54:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:57.132631 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d3e21-b526-4bef-ba16-cb9559af2a67 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:57.154284 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-5ebed452-d9a3-4c47-a774-c63a694110c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:57.154284 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:57.154284 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:57.154284 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:57.154284 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:57.154284 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:57.154284 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:57.154284 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:57.154284 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:57.154284 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:57.154284 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:57.154284 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:57.154284 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:54:57.154284 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:54:57.154284 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:54:57.155182 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:54:57.155182 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:57.155182 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:57.155182 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:57.155182 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:57.155182 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:57.155182 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:57.155182 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:57.155182 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:57.155182 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:57.155182 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:57.155182 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:57.155182 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:57.155182 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:57.155182 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:57.155182 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:57.155182 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:57.156103 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:54:57.156103 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:54:57.156103 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:57.156103 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:57.157194 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:57.157756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 283/593] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:54:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:57.159437 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebed452-d9a3-4c47-a774-c63a694110c2 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:57.216199 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:57.252859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d3e21-b526-4bef-ba16-cb9559af2a67 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:57.305475 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f62b30c-cf97-43f3-b30d-0dee0aaedf4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:57.306070 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebed452-d9a3-4c47-a774-c63a694110c2 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:57.306314 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f62b30c-cf97-43f3-b30d-0dee0aaedf4c None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:57.306602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f62b30c-cf97-43f3-b30d-0dee0aaedf4c None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:57.306883 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f62b30c-cf97-43f3-b30d-0dee0aaedf4c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:57.318001 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f62b30c-cf97-43f3-b30d-0dee0aaedf4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:57.319464 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f62b30c-cf97-43f3-b30d-0dee0aaedf4c None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:57.338184 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-1f62b30c-cf97-43f3-b30d-0dee0aaedf4c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:57.338184 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:57.338184 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:57.338184 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:57.338184 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:57.338184 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:57.338184 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:57.338184 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:57.338184 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:57.338184 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:57.338184 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:57.338184 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:57.338184 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:54:57.338184 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:54:57.338184 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:54:57.339110 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:54:57.339110 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:57.339110 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:57.339110 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:57.339110 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:57.339110 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:57.339110 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:57.339110 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:57.339110 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:57.339110 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:57.339110 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:57.339110 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:57.339110 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:57.339110 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:57.339110 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:57.339110 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:57.339110 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:57.340031 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:54:57.340031 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:54:57.340031 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:57.340031 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:57.340031 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 311/594] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:54:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:57.340763 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f62b30c-cf97-43f3-b30d-0dee0aaedf4c None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:57.347068 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c31442f-a896-45e6-87cf-9194eec5f18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:57.348713 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31442f-a896-45e6-87cf-9194eec5f18e None admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:57.349081 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31442f-a896-45e6-87cf-9194eec5f18e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:57.349433 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31442f-a896-45e6-87cf-9194eec5f18e None admin] PATH_INFO: `/v3/projects/7938b0346d4240ab89774327d09a89ae/users/a1bfa3e856ae46d48c82195f70aec24f/roles/750bca9c32ac4aa49adcbee30463f1fa` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:57.365855 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c31442f-a896-45e6-87cf-9194eec5f18e None admin] RBAC: Authorizing `identity:create_grant(project_id=7938b0346d4240ab89774327d09a89ae, user_id=a1bfa3e856ae46d48c82195f70aec24f, role_id=750bca9c32ac4aa49adcbee30463f1fa)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:57.367477 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c31442f-a896-45e6-87cf-9194eec5f18e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:57.424143 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:57.443975 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 284/595] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 6 16:54:57 2022] PUT /identity/v3/projects/7938b0346d4240ab89774327d09a89ae/users/a1bfa3e856ae46d48c82195f70aec24f/roles/750bca9c32ac4aa49adcbee30463f1fa => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 16:54:57.445134 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c31442f-a896-45e6-87cf-9194eec5f18e None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:57.459574 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f62b30c-cf97-43f3-b30d-0dee0aaedf4c None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:57.496809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d4eff6-c5be-4740-b74d-d1f2da3c0adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:57.497721 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4eff6-c5be-4740-b74d-d1f2da3c0adf None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:57.498155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4eff6-c5be-4740-b74d-d1f2da3c0adf None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:57.498609 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d4eff6-c5be-4740-b74d-d1f2da3c0adf None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:57.499481 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d4eff6-c5be-4740-b74d-d1f2da3c0adf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:57.499978 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d4eff6-c5be-4740-b74d-d1f2da3c0adf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:57.501538 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d4eff6-c5be-4740-b74d-d1f2da3c0adf None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:57.515554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 312/596] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:54:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:57.520789 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d4eff6-c5be-4740-b74d-d1f2da3c0adf None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:57.587534 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:57.592698 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c31442f-a896-45e6-87cf-9194eec5f18e None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:57.630992 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45be89c6-22b4-4b2d-be90-016beaefb878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:57.631757 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45be89c6-22b4-4b2d-be90-016beaefb878 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:57.632160 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45be89c6-22b4-4b2d-be90-016beaefb878 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:57.632534 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45be89c6-22b4-4b2d-be90-016beaefb878 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:57.633426 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45be89c6-22b4-4b2d-be90-016beaefb878 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:57.634602 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45be89c6-22b4-4b2d-be90-016beaefb878 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:57.636293 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45be89c6-22b4-4b2d-be90-016beaefb878 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:57.649929 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 285/597] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:54:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:57.655474 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45be89c6-22b4-4b2d-be90-016beaefb878 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:57.763128 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d4eff6-c5be-4740-b74d-d1f2da3c0adf None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:57.767322 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45be89c6-22b4-4b2d-be90-016beaefb878 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:57.794458 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:57.801615 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1431f24-396a-4301-a249-86ced4de7028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:57.803551 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1431f24-396a-4301-a249-86ced4de7028 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:57.804176 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447d458c-9907-4ee6-aff6-92d39b02962c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:57.804529 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1431f24-396a-4301-a249-86ced4de7028 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:57.807398 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d458c-9907-4ee6-aff6-92d39b02962c None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:57.807565 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1431f24-396a-4301-a249-86ced4de7028 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:57.807717 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d458c-9907-4ee6-aff6-92d39b02962c None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:57.807717 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447d458c-9907-4ee6-aff6-92d39b02962c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:57.814317 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447d458c-9907-4ee6-aff6-92d39b02962c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:57.815809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1431f24-396a-4301-a249-86ced4de7028 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:57.815931 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447d458c-9907-4ee6-aff6-92d39b02962c None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:57.817570 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1431f24-396a-4301-a249-86ced4de7028 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:57.831419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-447d458c-9907-4ee6-aff6-92d39b02962c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:54:57.831419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:57.831419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:57.831419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:57.831419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:57.831419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:57.831419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:57.831419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:57.831419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:57.831419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:57.831419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:57.831419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:57.831419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 16:54:57.831419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 16:54:57.831419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 16:54:57.832419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 16:54:57.832419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:57.832419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:57.832419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:57.832419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:57.832419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:57.832419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:57.832419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:57.832419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:57.832419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:57.832419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:57.832419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:57.832419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:57.832419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:57.832419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:57.832419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:57.832419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:57.833332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 16:54:57.833332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 16:54:57.833332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 16:54:57.833332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 16:54:57.833332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 16:54:57.833332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 16:54:57.833332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:54:57.833332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:57.833332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 286/598] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:54:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:57.837016 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-b1431f24-396a-4301-a249-86ced4de7028 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 06 16:54:57.837016 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:57.837016 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:57.837016 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:57.837016 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:57.837016 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:57.837016 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:57.837016 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:57.837016 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:57.837016 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:57.837016 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:57.837016 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:57.837016 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 16:54:57.837016 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 16:54:57.837016 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 16:54:57.837016 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 16:54:57.838200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:57.838200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:57.838200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 16:54:57.838200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:54:57.838200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 16:54:57.838200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:54:57.838200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:57.838200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:57.838200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:57.838200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:57.838200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:57.838200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:57.838200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:57.838200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:57.838200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:57.838200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:57.838200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:57.838200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:57.839164 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:57.839164 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:57.839164 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 16:54:57.839164 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 16:54:57.839164 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 16:54:57.839164 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 16:54:57.839164 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 16:54:57.839164 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 16:54:57.839164 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 06 16:54:57.839164 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:57.839686 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447d458c-9907-4ee6-aff6-92d39b02962c None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:57.841520 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 313/599] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 6 16:54:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:57.850192 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1431f24-396a-4301-a249-86ced4de7028 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:57.864025 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:57.958358 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1431f24-396a-4301-a249-86ced4de7028 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:57.974606 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447d458c-9907-4ee6-aff6-92d39b02962c None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:58.001559 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef9431a-c984-425b-800b-9696cd14d2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:58.002592 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef9431a-c984-425b-800b-9696cd14d2b0 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:58.003076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef9431a-c984-425b-800b-9696cd14d2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:58.003568 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef9431a-c984-425b-800b-9696cd14d2b0 None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:58.004643 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef9431a-c984-425b-800b-9696cd14d2b0 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:58.005183 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef9431a-c984-425b-800b-9696cd14d2b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:58.006843 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef9431a-c984-425b-800b-9696cd14d2b0 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:58.009728 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f51ac-2462-4d09-9c2a-4eda3bb0bf27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:58.010498 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f51ac-2462-4d09-9c2a-4eda3bb0bf27 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:58.011458 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f51ac-2462-4d09-9c2a-4eda3bb0bf27 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:58.011458 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f51ac-2462-4d09-9c2a-4eda3bb0bf27 None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:58.012127 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288f51ac-2462-4d09-9c2a-4eda3bb0bf27 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:58.012532 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288f51ac-2462-4d09-9c2a-4eda3bb0bf27 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:58.014069 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288f51ac-2462-4d09-9c2a-4eda3bb0bf27 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:58.026765 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 287/600] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 6 16:54:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:58.027902 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 314/601] 10.222.0.40 () {56 vars in 1139 bytes} [Wed Jul 6 16:54:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:58.032382 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b88255-d2ef-4d29-8500-5dfe8bde39a6 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:58.033383 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef9431a-c984-425b-800b-9696cd14d2b0 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:58.034056 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b88255-d2ef-4d29-8500-5dfe8bde39a6 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:58.034334 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b88255-d2ef-4d29-8500-5dfe8bde39a6 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:58.035838 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 288/602] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:58.036402 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b88255-d2ef-4d29-8500-5dfe8bde39a6 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:58.093361 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:58.132155 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:58.146669 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef9431a-c984-425b-800b-9696cd14d2b0 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:58.158536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b88255-d2ef-4d29-8500-5dfe8bde39a6 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:58.184162 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d62447-f7ba-4e49-b2e3-8e2a45ebef90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:58.185008 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d62447-f7ba-4e49-b2e3-8e2a45ebef90 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:58.185289 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d62447-f7ba-4e49-b2e3-8e2a45ebef90 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:58.185566 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d62447-f7ba-4e49-b2e3-8e2a45ebef90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:58.197232 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d62447-f7ba-4e49-b2e3-8e2a45ebef90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:58.198882 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d62447-f7ba-4e49-b2e3-8e2a45ebef90 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:58.201980 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819dc0f2-c7bb-400b-9644-0d855a629540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:58.203113 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819dc0f2-c7bb-400b-9644-0d855a629540 None admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:58.203582 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819dc0f2-c7bb-400b-9644-0d855a629540 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:58.204087 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819dc0f2-c7bb-400b-9644-0d855a629540 None admin] PATH_INFO: `/v3/projects/7938b0346d4240ab89774327d09a89ae/users/635a410f471c43109438d0c39874070b/roles/750bca9c32ac4aa49adcbee30463f1fa` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:58.221024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-12d62447-f7ba-4e49-b2e3-8e2a45ebef90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:58.221024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:58.221024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:58.221024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:58.221024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:58.221024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:58.221024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:58.221024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:58.221024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:58.221024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:58.221024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:58.221024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:58.221024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:54:58.221024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:54:58.221024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:54:58.221922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:54:58.221922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:58.221922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:58.221922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:58.221922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:58.221922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:58.221922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:58.221922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:58.221922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:58.221922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:58.221922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:58.221922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:58.221922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:58.221922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:58.221922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:58.221922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:58.221922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:58.222787 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:54:58.222787 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:54:58.222787 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:58.222787 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:58.223002 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819dc0f2-c7bb-400b-9644-0d855a629540 None admin] RBAC: Authorizing `identity:create_grant(project_id=7938b0346d4240ab89774327d09a89ae, user_id=635a410f471c43109438d0c39874070b, role_id=750bca9c32ac4aa49adcbee30463f1fa)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:58.223083 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 315/603] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:54:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:58.223153 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f9e71-376c-438a-907e-9917b4f9f02f None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:58.223452 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f9e71-376c-438a-907e-9917b4f9f02f None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:58.223744 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f9e71-376c-438a-907e-9917b4f9f02f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:58.224888 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819dc0f2-c7bb-400b-9644-0d855a629540 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:58.241788 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-9a8f9e71-376c-438a-907e-9917b4f9f02f None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:58.243552 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-9a8f9e71-376c-438a-907e-9917b4f9f02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:58.319555 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 316/604] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:58.320836 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f9e71-376c-438a-907e-9917b4f9f02f None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:58.354094 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:58.355239 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 289/605] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 6 16:54:58 2022] PUT /identity/v3/projects/7938b0346d4240ab89774327d09a89ae/users/635a410f471c43109438d0c39874070b/roles/750bca9c32ac4aa49adcbee30463f1fa => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 16:54:58.356049 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819dc0f2-c7bb-400b-9644-0d855a629540 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:58.437963 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8f9e71-376c-438a-907e-9917b4f9f02f None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:58.478447 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c4f72-e535-4ceb-8c94-07d8cd5407c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:58.479381 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c4f72-e535-4ceb-8c94-07d8cd5407c6 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:58.479876 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c4f72-e535-4ceb-8c94-07d8cd5407c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:58.480339 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c4f72-e535-4ceb-8c94-07d8cd5407c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:58.481263 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c4f72-e535-4ceb-8c94-07d8cd5407c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:58.481772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c4f72-e535-4ceb-8c94-07d8cd5407c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:58.483392 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c4f72-e535-4ceb-8c94-07d8cd5407c6 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:58.494242 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819dc0f2-c7bb-400b-9644-0d855a629540 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:58.501855 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 317/606] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:54:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:58.503632 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485ebc2-6634-42e3-bcc7-a70185a356b4 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:58.504151 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485ebc2-6634-42e3-bcc7-a70185a356b4 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:58.504603 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485ebc2-6634-42e3-bcc7-a70185a356b4 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:58.507168 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 318/607] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:58.508675 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e485ebc2-6634-42e3-bcc7-a70185a356b4 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:58.536817 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:58.544246 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed702f1-9c04-48ff-8dd1-56f70279d1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:58.545185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed702f1-9c04-48ff-8dd1-56f70279d1c9 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:58.545466 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed702f1-9c04-48ff-8dd1-56f70279d1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:58.553347 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed702f1-9c04-48ff-8dd1-56f70279d1c9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:58.556900 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed702f1-9c04-48ff-8dd1-56f70279d1c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:58.565642 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-8ed702f1-9c04-48ff-8dd1-56f70279d1c9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 16:54:58.565642 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:58.565642 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:58.565642 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:58.565642 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:58.565642 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:58.565642 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:58.565642 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:58.565642 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:58.565642 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:58.565642 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:58.565642 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:58.565642 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:58.565642 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:58.565642 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:58.565642 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:58.566754 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:58.566754 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:58.566754 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:58.566754 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:58.566754 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:58.566754 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:58.566754 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:58.566754 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:58.566754 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:58.566754 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:58.566754 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:58.566754 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:58.566754 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:58.566754 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:58.566754 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:58.566754 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:58.567884 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:58.567884 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:58.567884 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:58.567884 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:58.567884 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:58.567884 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:58.567884 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:58.567884 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:58.567884 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:58.567884 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:58.567884 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 16:54:58.567884 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:58.567884 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 290/608] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 6 16:54:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:58.568825 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503ea59-6a98-4742-99c9-708ad50ad65f None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:58.568825 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503ea59-6a98-4742-99c9-708ad50ad65f None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:58.568825 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5503ea59-6a98-4742-99c9-708ad50ad65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:58.589186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-5503ea59-6a98-4742-99c9-708ad50ad65f None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:58.590375 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-5503ea59-6a98-4742-99c9-708ad50ad65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:58.642480 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:58.655920 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e485ebc2-6634-42e3-bcc7-a70185a356b4 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:58.661892 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 291/609] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:54:58.662929 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503ea59-6a98-4742-99c9-708ad50ad65f None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:58.700849 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a790f-1fea-4475-a1c6-d09ae09be959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:58.701934 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a790f-1fea-4475-a1c6-d09ae09be959 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:58.702369 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a790f-1fea-4475-a1c6-d09ae09be959 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:58.702793 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645a790f-1fea-4475-a1c6-d09ae09be959 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:58.711449 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a790f-1fea-4475-a1c6-d09ae09be959 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:58.713751 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645a790f-1fea-4475-a1c6-d09ae09be959 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:58.732203 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-645a790f-1fea-4475-a1c6-d09ae09be959 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:54:58.732203 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:58.732203 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:58.732203 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:58.732203 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:58.732203 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:58.732203 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:58.732203 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:58.732203 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:58.732203 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:58.732203 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:58.732203 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:58.732203 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 16:54:58.732203 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 16:54:58.732203 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 16:54:58.733283 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 16:54:58.733283 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:58.733283 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:58.733283 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:58.733283 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:58.733283 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:58.733283 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:58.733283 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:58.733283 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:58.733283 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:58.733283 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:58.733283 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:58.733283 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:58.733283 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:58.733283 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:58.733283 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:58.733283 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:58.734451 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 16:54:58.734451 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 16:54:58.734451 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 16:54:58.734451 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 16:54:58.734451 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 16:54:58.734451 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 16:54:58.734451 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:54:58.734451 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:58.739760 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 319/610] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:54:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:58.739760 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a790f-1fea-4475-a1c6-d09ae09be959 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:58.779217 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5503ea59-6a98-4742-99c9-708ad50ad65f None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:58.802013 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:58.818269 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b0405f-106b-4dfb-b895-3196b65fea7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:58.819105 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0405f-106b-4dfb-b895-3196b65fea7f None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:58.819491 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0405f-106b-4dfb-b895-3196b65fea7f None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:58.819963 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0405f-106b-4dfb-b895-3196b65fea7f None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:58.821400 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b0405f-106b-4dfb-b895-3196b65fea7f None admin] RBAC: Adding query filter params (name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:58.821804 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b0405f-106b-4dfb-b895-3196b65fea7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:58.823333 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b0405f-106b-4dfb-b895-3196b65fea7f None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:58.831890 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 292/611] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 6 16:54:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:58.832875 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b0405f-106b-4dfb-b895-3196b65fea7f None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:58.866884 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645a790f-1fea-4475-a1c6-d09ae09be959 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:58.911605 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215bf1e-2d57-4126-ae3e-29977ffc0db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:58.912476 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215bf1e-2d57-4126-ae3e-29977ffc0db9 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:58.913406 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215bf1e-2d57-4126-ae3e-29977ffc0db9 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:58.913695 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215bf1e-2d57-4126-ae3e-29977ffc0db9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:58.929060 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9215bf1e-2d57-4126-ae3e-29977ffc0db9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:54:58.938486 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b0405f-106b-4dfb-b895-3196b65fea7f None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:58.939176 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-9215bf1e-2d57-4126-ae3e-29977ffc0db9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 16:54:58.939176 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:58.939176 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:58.939176 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:58.939176 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:58.939176 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:58.939176 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:58.939176 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:58.939176 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:58.939176 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:58.939176 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:58.939176 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:58.939176 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:54:58.939176 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:54:58.939176 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:54:58.939176 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:54:58.940097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:54:58.940097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:54:58.940097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:54:58.940097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:54:58.940097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:58.940097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:58.940097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:58.940097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:58.940097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:58.940097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:58.940097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:58.940097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:58.940097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:58.940097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:58.940097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:58.940097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:58.940927 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:58.940927 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:58.940927 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:58.940927 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:58.940927 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:54:58.940927 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:54:58.940927 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:54:58.940927 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:54:58.940927 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:54:58.940927 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:54:58.940927 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 16:54:58.940927 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:58.942011 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 320/612] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 6 16:54:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:58.947751 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215bf1e-2d57-4126-ae3e-29977ffc0db9 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:58.961275 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:58.986576 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bc5506-bb60-4739-a82b-c1b541cbc2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:58.987182 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc5506-bb60-4739-a82b-c1b541cbc2c2 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:58.987470 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc5506-bb60-4739-a82b-c1b541cbc2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:58.987779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bc5506-bb60-4739-a82b-c1b541cbc2c2 None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:58.988636 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bc5506-bb60-4739-a82b-c1b541cbc2c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:58.988950 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bc5506-bb60-4739-a82b-c1b541cbc2c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:58.990853 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bc5506-bb60-4739-a82b-c1b541cbc2c2 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:59.010585 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 293/613] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 6 16:54:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:59.012106 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bc5506-bb60-4739-a82b-c1b541cbc2c2 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:59.063154 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9215bf1e-2d57-4126-ae3e-29977ffc0db9 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:59.080082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:59.098475 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dfbc39-cbe2-4ef0-af53-8b09d86092cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:59.099272 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dfbc39-cbe2-4ef0-af53-8b09d86092cf None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:59.099557 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dfbc39-cbe2-4ef0-af53-8b09d86092cf None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:59.099885 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dfbc39-cbe2-4ef0-af53-8b09d86092cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:59.110662 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dfbc39-cbe2-4ef0-af53-8b09d86092cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:59.117472 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dfbc39-cbe2-4ef0-af53-8b09d86092cf None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:59.130529 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bc5506-bb60-4739-a82b-c1b541cbc2c2 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:59.130802 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-66dfbc39-cbe2-4ef0-af53-8b09d86092cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:59.130802 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:59.130802 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:59.130802 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:59.130802 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:59.130802 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:59.130802 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:59.130802 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:59.130802 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:59.130802 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:59.130802 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:59.130802 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:59.130802 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:54:59.130802 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:54:59.130802 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:54:59.133106 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:54:59.133106 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:59.133106 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:59.133106 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:59.133106 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:59.133106 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:59.133106 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:59.133106 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:59.133106 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:59.133106 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:59.133106 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:59.133106 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:59.133106 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:59.133106 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:59.133106 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:59.133106 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:59.133106 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:59.135815 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:54:59.135815 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:54:59.135815 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:59.135815 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:59.135815 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 321/614] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:54:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:59.135815 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dfbc39-cbe2-4ef0-af53-8b09d86092cf None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:59.174583 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eca1e1-53b8-4a90-bbf7-acbc9383ef8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:59.175258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eca1e1-53b8-4a90-bbf7-acbc9383ef8a None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:59.175528 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eca1e1-53b8-4a90-bbf7-acbc9383ef8a None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:59.175844 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04eca1e1-53b8-4a90-bbf7-acbc9383ef8a None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:59.177711 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eca1e1-53b8-4a90-bbf7-acbc9383ef8a None admin] RBAC: Adding query filter params (name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:59.178031 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eca1e1-53b8-4a90-bbf7-acbc9383ef8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:59.179409 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04eca1e1-53b8-4a90-bbf7-acbc9383ef8a None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:59.190416 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 294/615] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 6 16:54:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:59.192041 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eca1e1-53b8-4a90-bbf7-acbc9383ef8a None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:59.214386 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:59.255581 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dfbc39-cbe2-4ef0-af53-8b09d86092cf None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:59.292776 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba78494-8155-42af-a921-537f2d54f161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:59.293676 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba78494-8155-42af-a921-537f2d54f161 None admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:59.293964 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba78494-8155-42af-a921-537f2d54f161 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:59.294245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba78494-8155-42af-a921-537f2d54f161 None admin] PATH_INFO: `/v3/projects/7938b0346d4240ab89774327d09a89ae/users/5f479458a68b4daf9f3a7ae53d6c3763/roles/750bca9c32ac4aa49adcbee30463f1fa` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:59.305985 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04eca1e1-53b8-4a90-bbf7-acbc9383ef8a None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:59.308587 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:59.310625 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba78494-8155-42af-a921-537f2d54f161 None admin] RBAC: Authorizing `identity:create_grant(project_id=7938b0346d4240ab89774327d09a89ae, user_id=5f479458a68b4daf9f3a7ae53d6c3763, role_id=750bca9c32ac4aa49adcbee30463f1fa)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:59.312100 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba78494-8155-42af-a921-537f2d54f161 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:59.340747 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118a389-be57-42ca-8d98-ca3c9f302380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:59.341529 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118a389-be57-42ca-8d98-ca3c9f302380 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:59.342158 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118a389-be57-42ca-8d98-ca3c9f302380 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:59.342786 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1118a389-be57-42ca-8d98-ca3c9f302380 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:59.344028 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1118a389-be57-42ca-8d98-ca3c9f302380 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:59.344340 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1118a389-be57-42ca-8d98-ca3c9f302380 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:59.345697 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1118a389-be57-42ca-8d98-ca3c9f302380 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:59.358035 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 295/616] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:54:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:59.361856 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118a389-be57-42ca-8d98-ca3c9f302380 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:59.375886 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 322/617] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 6 16:54:59 2022] PUT /identity/v3/projects/7938b0346d4240ab89774327d09a89ae/users/5f479458a68b4daf9f3a7ae53d6c3763/roles/750bca9c32ac4aa49adcbee30463f1fa => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 16:54:59.376887 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba78494-8155-42af-a921-537f2d54f161 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:59.492299 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1118a389-be57-42ca-8d98-ca3c9f302380 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:59.536246 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb27ed-b418-4534-a787-2558aade177c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:59.537362 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb27ed-b418-4534-a787-2558aade177c None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:59.537868 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb27ed-b418-4534-a787-2558aade177c None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:59.538370 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb27ed-b418-4534-a787-2558aade177c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:59.552400 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fb27ed-b418-4534-a787-2558aade177c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:59.554304 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fb27ed-b418-4534-a787-2558aade177c None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:59.577034 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-95fb27ed-b418-4534-a787-2558aade177c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:59.577034 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:59.577034 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:59.577034 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:59.577034 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:59.577034 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:59.577034 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:59.577034 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:59.577034 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:59.577034 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:59.577034 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:59.577034 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:59.577034 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:54:59.577034 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:54:59.577034 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:54:59.578022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:54:59.578022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:59.578022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:59.578022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:59.578022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:59.578022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:59.578022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:59.578022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:59.578022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:59.578022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:59.578022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:59.578022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:59.578022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:59.578022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:59.578022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:59.578022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:59.578022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:59.578917 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:54:59.578917 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:54:59.578917 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:54:59.578917 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:54:59.581684 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 296/618] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:54:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:59.588449 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb27ed-b418-4534-a787-2558aade177c None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:59.610449 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:59.641531 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba78494-8155-42af-a921-537f2d54f161 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:59.659720 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:54:59.682202 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3169222-53b6-43c7-964c-53cf34fc9f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:59.683251 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3169222-53b6-43c7-964c-53cf34fc9f5e None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:59.683758 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3169222-53b6-43c7-964c-53cf34fc9f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:59.684200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3169222-53b6-43c7-964c-53cf34fc9f5e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:59.693443 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3169222-53b6-43c7-964c-53cf34fc9f5e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:59.695069 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3169222-53b6-43c7-964c-53cf34fc9f5e None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:59.711239 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-f3169222-53b6-43c7-964c-53cf34fc9f5e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 06 16:54:59.711239 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:54:59.711239 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:54:59.711239 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:54:59.711239 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:54:59.711239 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:54:59.711239 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:54:59.711239 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:54:59.711239 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:54:59.711239 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:54:59.711239 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:54:59.711239 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:54:59.711239 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 16:54:59.711239 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 16:54:59.711239 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 16:54:59.711239 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 16:54:59.712585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:54:59.712585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:54:59.712585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 16:54:59.712585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:54:59.712585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 16:54:59.712585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:54:59.712585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:54:59.712585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:54:59.712585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:54:59.712585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:54:59.712585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:54:59.712585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:54:59.712585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:54:59.712585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:54:59.712585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:54:59.712585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:54:59.712585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:54:59.712585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:54:59.713802 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:54:59.713802 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:54:59.713802 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 16:54:59.713802 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 16:54:59.713802 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 16:54:59.713802 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 16:54:59.713802 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 16:54:59.713802 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 16:54:59.713802 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 06 16:54:59.713802 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:54:59.716160 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 323/619] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 6 16:54:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:54:59.722354 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3169222-53b6-43c7-964c-53cf34fc9f5e None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:59.819839 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb27ed-b418-4534-a787-2558aade177c None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:59.833060 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3169222-53b6-43c7-964c-53cf34fc9f5e None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:54:59.872291 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa8ea5c-64c1-4117-8932-148c53d5fe3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:59.873031 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8ea5c-64c1-4117-8932-148c53d5fe3e None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:59.873330 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8ea5c-64c1-4117-8932-148c53d5fe3e None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:59.873647 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa8ea5c-64c1-4117-8932-148c53d5fe3e None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:59.874545 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa8ea5c-64c1-4117-8932-148c53d5fe3e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:59.874859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa8ea5c-64c1-4117-8932-148c53d5fe3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:59.876496 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa8ea5c-64c1-4117-8932-148c53d5fe3e None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:59.890942 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc1d578-90b3-4e09-bec6-fb8226d5d170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:54:59.891965 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1d578-90b3-4e09-bec6-fb8226d5d170 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:59.892434 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1d578-90b3-4e09-bec6-fb8226d5d170 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:59.892890 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1d578-90b3-4e09-bec6-fb8226d5d170 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:59.893819 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc1d578-90b3-4e09-bec6-fb8226d5d170 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:54:59.894306 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc1d578-90b3-4e09-bec6-fb8226d5d170 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:54:59.895881 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc1d578-90b3-4e09-bec6-fb8226d5d170 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:54:59.896474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 324/620] 10.222.0.40 () {56 vars in 1135 bytes} [Wed Jul 6 16:54:59 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:59.902009 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6c998-eb9d-4993-bd1a-1207ea8a0a6a None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:59.902009 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6c998-eb9d-4993-bd1a-1207ea8a0a6a None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:59.902009 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6c998-eb9d-4993-bd1a-1207ea8a0a6a None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:59.902009 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 325/621] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:54:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:54:59.904607 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d6c998-eb9d-4993-bd1a-1207ea8a0a6a None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:54:59.914659 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 297/622] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:54:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:54:59.916305 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef92f44a-d19c-4b8f-943c-b254e3dba716 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:54:59.916866 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef92f44a-d19c-4b8f-943c-b254e3dba716 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:54:59.917149 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef92f44a-d19c-4b8f-943c-b254e3dba716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:54:59.934081 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-ef92f44a-d19c-4b8f-943c-b254e3dba716 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:54:59.935281 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ef92f44a-d19c-4b8f-943c-b254e3dba716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:54:59.938332 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:00.024385 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 298/623] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:54:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:55:00.024385 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef92f44a-d19c-4b8f-943c-b254e3dba716 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:00.032067 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:00.048523 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d6c998-eb9d-4993-bd1a-1207ea8a0a6a None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:00.091440 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5a186-2b2c-4f0a-a00e-ee3dda990ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:00.092227 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5a186-2b2c-4f0a-a00e-ee3dda990ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:00.093507 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5a186-2b2c-4f0a-a00e-ee3dda990ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:00.093794 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e5a186-2b2c-4f0a-a00e-ee3dda990ac6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:00.105396 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e5a186-2b2c-4f0a-a00e-ee3dda990ac6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:00.106953 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e5a186-2b2c-4f0a-a00e-ee3dda990ac6 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:00.129047 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef92f44a-d19c-4b8f-943c-b254e3dba716 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:00.133130 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-30e5a186-2b2c-4f0a-a00e-ee3dda990ac6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:00.133130 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:00.133130 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:00.133130 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:00.133130 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:00.133130 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:00.133130 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:00.133130 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:00.133130 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:00.133130 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:00.133130 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:00.133130 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:00.133130 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:55:00.133130 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:55:00.133130 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:55:00.135322 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:55:00.135322 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:00.135322 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:00.135322 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:00.135322 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:00.135322 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:00.135322 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:00.135322 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:00.135322 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:00.135322 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:00.135322 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:00.135322 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:00.135322 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:00.135322 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:00.135322 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:00.135322 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:00.135322 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:00.137151 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:55:00.137151 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:55:00.137151 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:00.137151 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:00.137151 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 326/624] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:54:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:00.137151 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5a186-2b2c-4f0a-a00e-ee3dda990ac6 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:00.178394 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd6f24c-ac01-4ac4-aefb-154e5a91d7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:00.179413 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6f24c-ac01-4ac4-aefb-154e5a91d7e6 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:00.179704 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6f24c-ac01-4ac4-aefb-154e5a91d7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:00.180039 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd6f24c-ac01-4ac4-aefb-154e5a91d7e6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:00.188191 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd6f24c-ac01-4ac4-aefb-154e5a91d7e6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:00.189917 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd6f24c-ac01-4ac4-aefb-154e5a91d7e6 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:00.206784 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-3dd6f24c-ac01-4ac4-aefb-154e5a91d7e6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 06 16:55:00.206784 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:00.206784 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:00.206784 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:00.206784 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:00.206784 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:00.206784 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:00.206784 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:00.206784 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:00.206784 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:00.206784 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:00.206784 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:00.206784 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 16:55:00.206784 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 16:55:00.206784 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 16:55:00.206784 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 16:55:00.209001 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:00.209001 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:00.209001 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 16:55:00.209001 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:00.209001 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 16:55:00.209001 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:00.209001 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:00.209001 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:00.209001 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:00.209001 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:00.209001 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:00.209001 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:00.209001 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:00.209001 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:00.209001 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:00.209001 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:00.209001 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:00.209001 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:00.211182 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:00.211182 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:00.211182 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 16:55:00.211182 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 16:55:00.211182 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 16:55:00.211182 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 16:55:00.211182 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 16:55:00.211182 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 16:55:00.211182 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 06 16:55:00.211182 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:00.211182 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 299/625] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Jul 6 16:55:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:00.211182 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd6f24c-ac01-4ac4-aefb-154e5a91d7e6 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:00.245882 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:00.264128 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e5a186-2b2c-4f0a-a00e-ee3dda990ac6 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:00.285209 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:00.305157 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ac1c96-f3f9-48bb-9406-cde21d5ef578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:00.305976 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac1c96-f3f9-48bb-9406-cde21d5ef578 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:00.306264 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac1c96-f3f9-48bb-9406-cde21d5ef578 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:00.306547 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac1c96-f3f9-48bb-9406-cde21d5ef578 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:00.319364 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ac1c96-f3f9-48bb-9406-cde21d5ef578 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:55:00.325886 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd6f24c-ac01-4ac4-aefb-154e5a91d7e6 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:00.328284 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-14ac1c96-f3f9-48bb-9406-cde21d5ef578 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 16:55:00.328284 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:00.328284 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:00.328284 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:00.328284 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:00.328284 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:00.328284 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:00.328284 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:00.328284 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:00.328284 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:00.328284 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:00.328284 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:00.328284 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:55:00.328284 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:55:00.328284 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:55:00.328284 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:55:00.329222 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:55:00.329222 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:55:00.329222 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:55:00.329222 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:55:00.329222 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:00.329222 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:00.329222 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:00.329222 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:00.329222 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:00.329222 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:00.329222 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:00.329222 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:00.329222 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:00.329222 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:00.329222 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:00.329222 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:00.330052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:00.330052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:00.330052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:00.330052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:00.330052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:55:00.330052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:55:00.330052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:55:00.330052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:55:00.330052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:55:00.330052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:55:00.330052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 16:55:00.330052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:00.330052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 327/626] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 6 16:55:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:00.330734 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ac1c96-f3f9-48bb-9406-cde21d5ef578 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:00.367379 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c4b125-fac2-4c9e-8e2d-c62249746f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:00.368463 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4b125-fac2-4c9e-8e2d-c62249746f3e None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:00.368924 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4b125-fac2-4c9e-8e2d-c62249746f3e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:00.369350 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c4b125-fac2-4c9e-8e2d-c62249746f3e None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:00.370324 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c4b125-fac2-4c9e-8e2d-c62249746f3e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:00.370912 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c4b125-fac2-4c9e-8e2d-c62249746f3e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:00.372470 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c4b125-fac2-4c9e-8e2d-c62249746f3e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:00.391048 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 300/627] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:55:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:00.392194 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c4b125-fac2-4c9e-8e2d-c62249746f3e None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:00.446751 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ac1c96-f3f9-48bb-9406-cde21d5ef578 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:00.487280 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a627b1c3-aa19-4367-b049-1d45861779f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:00.488417 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a627b1c3-aa19-4367-b049-1d45861779f3 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:00.488908 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a627b1c3-aa19-4367-b049-1d45861779f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:00.489654 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a627b1c3-aa19-4367-b049-1d45861779f3 None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:00.491132 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a627b1c3-aa19-4367-b049-1d45861779f3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:00.491688 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a627b1c3-aa19-4367-b049-1d45861779f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:00.494849 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a627b1c3-aa19-4367-b049-1d45861779f3 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:00.503860 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c4b125-fac2-4c9e-8e2d-c62249746f3e None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:00.520120 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 328/628] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Jul 6 16:55:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:00.521012 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a627b1c3-aa19-4367-b049-1d45861779f3 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:00.525599 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:00.539225 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:00.548739 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8f106d-7e9c-45ed-830b-04e56f0e1170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:00.549715 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f106d-7e9c-45ed-830b-04e56f0e1170 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:00.550151 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f106d-7e9c-45ed-830b-04e56f0e1170 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:00.550577 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f106d-7e9c-45ed-830b-04e56f0e1170 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:00.551475 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8f106d-7e9c-45ed-830b-04e56f0e1170 None admin] RBAC: Adding query filter params (name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:00.551968 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8f106d-7e9c-45ed-830b-04e56f0e1170 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:00.553509 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8f106d-7e9c-45ed-830b-04e56f0e1170 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:00.564793 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 301/629] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 6 16:55:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:00.566025 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8f106d-7e9c-45ed-830b-04e56f0e1170 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:00.645521 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a627b1c3-aa19-4367-b049-1d45861779f3 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:00.684519 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8f106d-7e9c-45ed-830b-04e56f0e1170 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:00.685541 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b3ae7-904c-43aa-a02e-b9aaebf2846e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:00.686355 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b3ae7-904c-43aa-a02e-b9aaebf2846e None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:00.686641 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b3ae7-904c-43aa-a02e-b9aaebf2846e None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:00.686921 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b3ae7-904c-43aa-a02e-b9aaebf2846e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:00.694561 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0b3ae7-904c-43aa-a02e-b9aaebf2846e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:00.696605 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0b3ae7-904c-43aa-a02e-b9aaebf2846e None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:00.712011 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-9a0b3ae7-904c-43aa-a02e-b9aaebf2846e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:00.712011 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:00.712011 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:00.712011 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:00.712011 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:00.712011 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:00.712011 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:00.712011 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:00.712011 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:00.712011 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:00.712011 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:00.712011 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:00.712011 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 16:55:00.712011 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 16:55:00.712011 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 16:55:00.712981 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 16:55:00.712981 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:00.712981 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:00.712981 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:00.712981 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:00.712981 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:00.712981 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:00.712981 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:00.712981 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:00.712981 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:00.712981 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:00.712981 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:00.712981 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:00.712981 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:00.712981 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:00.712981 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:00.712981 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:00.713862 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 16:55:00.713862 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 16:55:00.713862 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 16:55:00.713862 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 16:55:00.713862 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 16:55:00.713862 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 16:55:00.713862 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:00.713862 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:00.713862 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 329/630] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:55:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:00.715038 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b3ae7-904c-43aa-a02e-b9aaebf2846e None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:00.731044 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a2bc2d-97e7-4a48-aa65-b90304da729f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:00.731044 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2bc2d-97e7-4a48-aa65-b90304da729f None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:00.731044 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2bc2d-97e7-4a48-aa65-b90304da729f None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:00.731044 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2bc2d-97e7-4a48-aa65-b90304da729f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:00.790216 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:00.795171 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:00.810759 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a2bc2d-97e7-4a48-aa65-b90304da729f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:00.812315 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a2bc2d-97e7-4a48-aa65-b90304da729f None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:00.837135 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b3ae7-904c-43aa-a02e-b9aaebf2846e None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:00.880686 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb96dbaa-e3e9-4094-aba4-214dca20e6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:00.881491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96dbaa-e3e9-4094-aba4-214dca20e6df None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:00.881778 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96dbaa-e3e9-4094-aba4-214dca20e6df None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:00.882056 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb96dbaa-e3e9-4094-aba4-214dca20e6df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:00.894000 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb96dbaa-e3e9-4094-aba4-214dca20e6df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:00.895609 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb96dbaa-e3e9-4094-aba4-214dca20e6df None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:00.914345 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-cb96dbaa-e3e9-4094-aba4-214dca20e6df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:00.914345 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:00.914345 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:00.914345 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:00.914345 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:00.914345 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:00.914345 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:00.914345 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:00.914345 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:00.914345 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:00.914345 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:00.914345 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:00.914345 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:55:00.914345 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:55:00.914345 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:55:00.915834 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:55:00.915834 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:00.915834 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:00.915834 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:00.915834 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:00.915834 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:00.915834 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:00.915834 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:00.915834 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:00.915834 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:00.915834 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:00.915834 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:00.915834 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:00.915834 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:00.915834 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:00.915834 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:00.915834 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:00.916878 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:55:00.916878 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:55:00.916878 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:00.916878 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:00.916878 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 330/631] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:00.917338 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7c430-9902-4621-ad00-59bb3991cd83 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:00.917628 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7c430-9902-4621-ad00-59bb3991cd83 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:00.917906 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc7c430-9902-4621-ad00-59bb3991cd83 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:00.919397 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 331/632] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:00.920057 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7c430-9902-4621-ad00-59bb3991cd83 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:00.945187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 302/633] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:55:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 16:55:00.945187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a2bc2d-97e7-4a48-aa65-b90304da729f None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:01.032952 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:01.042391 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc7c430-9902-4621-ad00-59bb3991cd83 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:01.045739 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a2bc2d-97e7-4a48-aa65-b90304da729f None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:01.058429 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:01.078868 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da8a130-294a-45a5-9235-aef42feda942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:01.079536 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da8a130-294a-45a5-9235-aef42feda942 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:01.079850 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da8a130-294a-45a5-9235-aef42feda942 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:01.080127 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da8a130-294a-45a5-9235-aef42feda942 None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:01.080361 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8012c64-c45f-49d9-804a-251af728de26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:01.080914 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da8a130-294a-45a5-9235-aef42feda942 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:01.081013 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8012c64-c45f-49d9-804a-251af728de26 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:01.081223 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da8a130-294a-45a5-9235-aef42feda942 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:01.081330 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8012c64-c45f-49d9-804a-251af728de26 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:01.081547 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8012c64-c45f-49d9-804a-251af728de26 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:01.082262 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8012c64-c45f-49d9-804a-251af728de26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:01.082565 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8012c64-c45f-49d9-804a-251af728de26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:01.082657 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da8a130-294a-45a5-9235-aef42feda942 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:01.083940 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8012c64-c45f-49d9-804a-251af728de26 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:01.099521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 303/634] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:55:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:01.099521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ba6707-5d30-4252-ae8a-241ee40749e7 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:01.099521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ba6707-5d30-4252-ae8a-241ee40749e7 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:01.099521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ba6707-5d30-4252-ae8a-241ee40749e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:01.099945 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 332/635] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 6 16:55:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:01.100735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da8a130-294a-45a5-9235-aef42feda942 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:01.118736 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-e2ba6707-5d30-4252-ae8a-241ee40749e7 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:01.119973 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e2ba6707-5d30-4252-ae8a-241ee40749e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:01.199750 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 304/636] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:55:01.199750 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ba6707-5d30-4252-ae8a-241ee40749e7 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:01.222271 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da8a130-294a-45a5-9235-aef42feda942 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:01.266653 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de570504-92fa-481e-ba15-df9624a116ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:01.267454 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de570504-92fa-481e-ba15-df9624a116ef None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:01.267761 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de570504-92fa-481e-ba15-df9624a116ef None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:01.268038 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de570504-92fa-481e-ba15-df9624a116ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:01.280423 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:01.281092 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de570504-92fa-481e-ba15-df9624a116ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:01.282690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de570504-92fa-481e-ba15-df9624a116ef None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:01.300966 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-de570504-92fa-481e-ba15-df9624a116ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:01.300966 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:01.300966 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:01.300966 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:01.300966 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:01.300966 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:01.300966 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:01.300966 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:01.300966 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:01.300966 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:01.300966 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:01.300966 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:01.300966 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:55:01.300966 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:55:01.300966 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:55:01.302141 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:55:01.302141 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:01.302141 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:01.302141 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:01.302141 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:01.302141 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:01.302141 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:01.302141 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:01.302141 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:01.302141 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:01.302141 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:01.302141 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:01.302141 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:01.302141 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:01.302141 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:01.302141 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:01.302141 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:01.303207 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:55:01.303207 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:55:01.303207 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:01.303207 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:01.304956 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ba6707-5d30-4252-ae8a-241ee40749e7 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:01.306663 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 333/637] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:01.307480 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de570504-92fa-481e-ba15-df9624a116ef None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:01.335719 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:01.342529 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e2bcbb-f89d-4dc8-86c0-872f798c1840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:01.343331 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e2bcbb-f89d-4dc8-86c0-872f798c1840 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:01.343669 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e2bcbb-f89d-4dc8-86c0-872f798c1840 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:01.343987 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e2bcbb-f89d-4dc8-86c0-872f798c1840 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:01.351565 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e2bcbb-f89d-4dc8-86c0-872f798c1840 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:01.353606 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e2bcbb-f89d-4dc8-86c0-872f798c1840 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:01.371082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-f2e2bcbb-f89d-4dc8-86c0-872f798c1840 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 06 16:55:01.371082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:01.371082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:01.371082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:01.371082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:01.371082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:01.371082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:01.371082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:01.371082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:01.371082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:01.371082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:01.371082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:01.371082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 16:55:01.371082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 16:55:01.371082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 16:55:01.371082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 16:55:01.372134 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:01.372134 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:01.372134 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 16:55:01.372134 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:01.372134 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 16:55:01.372134 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:01.372134 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:01.372134 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:01.372134 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:01.372134 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:01.372134 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:01.372134 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:01.372134 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:01.372134 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:01.372134 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:01.372134 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:01.372134 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:01.372134 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:01.373165 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:01.373165 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:01.373165 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 16:55:01.373165 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 16:55:01.373165 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 16:55:01.373165 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 16:55:01.373165 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 16:55:01.373165 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 16:55:01.373165 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 06 16:55:01.373165 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:01.373165 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 305/638] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:55:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:01.373165 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e2bcbb-f89d-4dc8-86c0-872f798c1840 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:01.415422 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de570504-92fa-481e-ba15-df9624a116ef None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:01.451014 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6e8110-a5b2-4324-b935-ba2d1917fe3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:01.451846 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e8110-a5b2-4324-b935-ba2d1917fe3a None admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:01.452127 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e8110-a5b2-4324-b935-ba2d1917fe3a None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:01.452806 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6e8110-a5b2-4324-b935-ba2d1917fe3a None admin] PATH_INFO: `/v3/projects/7938b0346d4240ab89774327d09a89ae/users/75acacd30e9a4cf2ad58b072b49e3f05/roles/750bca9c32ac4aa49adcbee30463f1fa` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:01.468609 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6e8110-a5b2-4324-b935-ba2d1917fe3a None admin] RBAC: Authorizing `identity:create_grant(project_id=7938b0346d4240ab89774327d09a89ae, user_id=75acacd30e9a4cf2ad58b072b49e3f05, role_id=750bca9c32ac4aa49adcbee30463f1fa)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:01.470148 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6e8110-a5b2-4324-b935-ba2d1917fe3a None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:01.478845 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e2bcbb-f89d-4dc8-86c0-872f798c1840 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:01.512377 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5c29b7-0c44-4b8f-8bec-17795a1bce4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:01.513326 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c29b7-0c44-4b8f-8bec-17795a1bce4c None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:01.513616 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c29b7-0c44-4b8f-8bec-17795a1bce4c None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:01.513907 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5c29b7-0c44-4b8f-8bec-17795a1bce4c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:01.520641 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:01.524636 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5c29b7-0c44-4b8f-8bec-17795a1bce4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:55:01.532454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-ee5c29b7-0c44-4b8f-8bec-17795a1bce4c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 16:55:01.532454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:01.532454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:01.532454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:01.532454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:01.532454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:01.532454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:01.532454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:01.532454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:01.532454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:01.532454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:01.532454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:01.532454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:55:01.532454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:55:01.532454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:55:01.532454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:55:01.533531 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:55:01.533531 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:55:01.533531 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:55:01.533531 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:55:01.533531 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:01.533531 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:01.533531 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:01.533531 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:01.533531 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:01.533531 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:01.533531 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:01.533531 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:01.533531 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:01.533531 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:01.533531 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:01.533531 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:01.534375 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:01.534375 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:01.534375 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:01.534375 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:01.534375 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:55:01.534375 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:55:01.534375 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:55:01.534375 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:55:01.534375 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:55:01.534375 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:55:01.534375 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 16:55:01.534375 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:01.534375 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 306/639] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 6 16:55:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:01.534375 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5c29b7-0c44-4b8f-8bec-17795a1bce4c None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:01.548978 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 334/640] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 6 16:55:01 2022] PUT /identity/v3/projects/7938b0346d4240ab89774327d09a89ae/users/75acacd30e9a4cf2ad58b072b49e3f05/roles/750bca9c32ac4aa49adcbee30463f1fa => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 16:55:01.550094 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6e8110-a5b2-4324-b935-ba2d1917fe3a None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:01.642301 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:01.669832 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6e8110-a5b2-4324-b935-ba2d1917fe3a None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:01.671540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5c29b7-0c44-4b8f-8bec-17795a1bce4c None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:01.709959 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1107bc6-d4cc-4305-8411-51b17fc7ef26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:01.710650 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1107bc6-d4cc-4305-8411-51b17fc7ef26 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:01.710939 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1107bc6-d4cc-4305-8411-51b17fc7ef26 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:01.711223 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1107bc6-d4cc-4305-8411-51b17fc7ef26 None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:01.712685 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1107bc6-d4cc-4305-8411-51b17fc7ef26 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:01.713404 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1107bc6-d4cc-4305-8411-51b17fc7ef26 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:01.715495 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1107bc6-d4cc-4305-8411-51b17fc7ef26 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:01.723761 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdbd7cf-b898-49e4-963a-ebd43eac3c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:01.724502 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdbd7cf-b898-49e4-963a-ebd43eac3c07 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:01.724792 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdbd7cf-b898-49e4-963a-ebd43eac3c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:01.725018 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdbd7cf-b898-49e4-963a-ebd43eac3c07 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:01.725748 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdbd7cf-b898-49e4-963a-ebd43eac3c07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:01.726056 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdbd7cf-b898-49e4-963a-ebd43eac3c07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:01.727496 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdbd7cf-b898-49e4-963a-ebd43eac3c07 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:01.744358 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 307/641] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:55:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:01.745143 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdbd7cf-b898-49e4-963a-ebd43eac3c07 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:01.746790 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 335/642] 10.222.0.40 () {56 vars in 1137 bytes} [Wed Jul 6 16:55:01 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:01.748463 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f0633-7500-48e3-bdaf-7b95fa24fde5 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:01.749732 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f0633-7500-48e3-bdaf-7b95fa24fde5 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:01.750149 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185f0633-7500-48e3-bdaf-7b95fa24fde5 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:01.751934 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 336/643] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:01.752402 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f0633-7500-48e3-bdaf-7b95fa24fde5 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:01.808824 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:01.860688 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185f0633-7500-48e3-bdaf-7b95fa24fde5 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:01.861360 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdbd7cf-b898-49e4-963a-ebd43eac3c07 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:01.898153 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee8dff-ea12-4792-ad4d-e1d333555bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:01.899590 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee8dff-ea12-4792-ad4d-e1d333555bb3 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:01.899966 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee8dff-ea12-4792-ad4d-e1d333555bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:01.900304 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee8dff-ea12-4792-ad4d-e1d333555bb3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:01.901156 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b84c22e-6723-4f58-b96b-4ac396158002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:01.901900 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84c22e-6723-4f58-b96b-4ac396158002 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:01.902367 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84c22e-6723-4f58-b96b-4ac396158002 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:01.902614 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84c22e-6723-4f58-b96b-4ac396158002 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:01.902748 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:01.903559 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b84c22e-6723-4f58-b96b-4ac396158002 None admin] RBAC: Adding query filter params (name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:01.904007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b84c22e-6723-4f58-b96b-4ac396158002 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:01.905469 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b84c22e-6723-4f58-b96b-4ac396158002 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:01.908190 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bee8dff-ea12-4792-ad4d-e1d333555bb3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:01.909625 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bee8dff-ea12-4792-ad4d-e1d333555bb3 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:01.918879 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 308/644] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 6 16:55:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:01.918879 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b84c22e-6723-4f58-b96b-4ac396158002 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:01.927752 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-6bee8dff-ea12-4792-ad4d-e1d333555bb3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:01.927752 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:01.927752 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:01.927752 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:01.927752 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:01.927752 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:01.927752 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:01.927752 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:01.927752 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:01.927752 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:01.927752 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:01.927752 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:01.927752 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 16:55:01.927752 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 16:55:01.927752 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 16:55:01.928722 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 16:55:01.928722 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:01.928722 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:01.928722 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:01.928722 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:01.928722 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:01.928722 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:01.928722 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:01.928722 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:01.928722 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:01.928722 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:01.928722 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:01.928722 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:01.928722 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:01.928722 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:01.928722 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:01.928722 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:01.929777 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 16:55:01.929777 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 16:55:01.929777 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 16:55:01.929777 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 16:55:01.929777 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 16:55:01.929777 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 16:55:01.929777 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:01.929777 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:01.929777 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 337/645] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:55:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:01.930915 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203bf018-e44d-4c93-82bf-a34fd6e44b42 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:01.930915 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203bf018-e44d-4c93-82bf-a34fd6e44b42 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:01.931055 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203bf018-e44d-4c93-82bf-a34fd6e44b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:01.948751 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-203bf018-e44d-4c93-82bf-a34fd6e44b42 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:01.949940 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-203bf018-e44d-4c93-82bf-a34fd6e44b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:02.019843 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 338/646] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:55:02.028576 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203bf018-e44d-4c93-82bf-a34fd6e44b42 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:02.033648 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b84c22e-6723-4f58-b96b-4ac396158002 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:02.053680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:02.068416 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f0647-3256-4968-b390-b62f9644a2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:02.071834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f0647-3256-4968-b390-b62f9644a2ad None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:02.071834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f0647-3256-4968-b390-b62f9644a2ad None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:02.071834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6f0647-3256-4968-b390-b62f9644a2ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:02.135326 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203bf018-e44d-4c93-82bf-a34fd6e44b42 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:02.142467 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6f0647-3256-4968-b390-b62f9644a2ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:02.143964 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6f0647-3256-4968-b390-b62f9644a2ad None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:02.158815 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:02.170364 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb35c0-6b78-44a3-b022-26d982d2bc8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:02.171158 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb35c0-6b78-44a3-b022-26d982d2bc8d None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:02.171439 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb35c0-6b78-44a3-b022-26d982d2bc8d None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:02.171712 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb35c0-6b78-44a3-b022-26d982d2bc8d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:02.186973 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cb35c0-6b78-44a3-b022-26d982d2bc8d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:02.188655 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cb35c0-6b78-44a3-b022-26d982d2bc8d None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:02.207577 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-13cb35c0-6b78-44a3-b022-26d982d2bc8d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:02.207577 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:02.207577 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:02.207577 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:02.207577 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:02.207577 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:02.207577 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:02.207577 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:02.207577 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:02.207577 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:02.207577 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:02.207577 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:02.207577 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:55:02.207577 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:55:02.207577 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:55:02.208712 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:55:02.208712 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:02.208712 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:02.208712 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:02.208712 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:02.208712 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:02.208712 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:02.208712 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:02.208712 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:02.208712 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:02.208712 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:02.208712 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:02.208712 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:02.208712 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:02.208712 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:02.208712 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:02.208712 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:02.209810 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:55:02.209810 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:55:02.209810 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:02.209810 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:02.216943 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 339/647] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:02.216943 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb35c0-6b78-44a3-b022-26d982d2bc8d None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:02.258815 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 309/648] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:55:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 16:55:02.260505 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f0647-3256-4968-b390-b62f9644a2ad None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:02.284646 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:02.331762 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cb35c0-6b78-44a3-b022-26d982d2bc8d None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:02.364007 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49377ba-6369-4387-abf5-adb6a04fc0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:02.364673 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49377ba-6369-4387-abf5-adb6a04fc0ce None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:02.364785 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6f0647-3256-4968-b390-b62f9644a2ad None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:02.364953 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49377ba-6369-4387-abf5-adb6a04fc0ce None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:02.365229 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49377ba-6369-4387-abf5-adb6a04fc0ce None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:02.366014 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49377ba-6369-4387-abf5-adb6a04fc0ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:02.366322 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49377ba-6369-4387-abf5-adb6a04fc0ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:02.368375 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49377ba-6369-4387-abf5-adb6a04fc0ce None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:02.384953 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 340/649] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 6 16:55:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:02.384953 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49377ba-6369-4387-abf5-adb6a04fc0ce None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:02.403940 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c415c245-047c-407b-accb-8d296147df16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:02.404737 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415c245-047c-407b-accb-8d296147df16 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:02.405019 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415c245-047c-407b-accb-8d296147df16 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:02.405294 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c415c245-047c-407b-accb-8d296147df16 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:02.415668 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c415c245-047c-407b-accb-8d296147df16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:55:02.419637 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:02.423331 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-c415c245-047c-407b-accb-8d296147df16 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 16:55:02.423331 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:02.423331 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:02.423331 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:02.423331 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:02.423331 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:02.423331 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:02.423331 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:02.423331 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:02.423331 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:02.423331 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:02.423331 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:02.423331 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:55:02.423331 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:55:02.423331 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:55:02.423331 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:55:02.424340 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:55:02.424340 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:55:02.424340 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:55:02.424340 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:55:02.424340 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:02.424340 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:02.424340 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:02.424340 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:02.424340 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:02.424340 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:02.424340 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:02.424340 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:02.424340 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:02.424340 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:02.424340 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:02.424340 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:02.425227 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:02.425227 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:02.425227 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:02.425227 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:02.425227 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:55:02.425227 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:55:02.425227 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:55:02.425227 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:55:02.425227 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:55:02.425227 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:55:02.425227 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 16:55:02.425227 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:02.425227 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 310/650] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 6 16:55:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:02.425992 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c415c245-047c-407b-accb-8d296147df16 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:02.485853 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49377ba-6369-4387-abf5-adb6a04fc0ce None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:02.519025 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b721c9-e856-4dd2-9433-c8644c98b9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:02.519691 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b721c9-e856-4dd2-9433-c8644c98b9d5 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:02.520008 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b721c9-e856-4dd2-9433-c8644c98b9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:02.520288 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b721c9-e856-4dd2-9433-c8644c98b9d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:02.521544 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b721c9-e856-4dd2-9433-c8644c98b9d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:02.521677 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b721c9-e856-4dd2-9433-c8644c98b9d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:02.522980 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b721c9-e856-4dd2-9433-c8644c98b9d5 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:02.533303 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c415c245-047c-407b-accb-8d296147df16 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:02.538617 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 341/651] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:55:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:02.538617 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b721c9-e856-4dd2-9433-c8644c98b9d5 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:02.567054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:02.572877 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb82970b-5dc9-4e65-b47b-7210dda4d6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:02.573675 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82970b-5dc9-4e65-b47b-7210dda4d6eb None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:02.573958 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82970b-5dc9-4e65-b47b-7210dda4d6eb None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:02.574229 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82970b-5dc9-4e65-b47b-7210dda4d6eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:02.587745 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb82970b-5dc9-4e65-b47b-7210dda4d6eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:02.593517 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb82970b-5dc9-4e65-b47b-7210dda4d6eb None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:02.606632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-cb82970b-5dc9-4e65-b47b-7210dda4d6eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:02.606632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:02.606632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:02.606632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:02.606632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:02.606632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:02.606632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:02.606632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:02.606632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:02.606632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:02.606632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:02.606632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:02.606632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:55:02.606632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:55:02.606632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:55:02.607536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:55:02.607536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:02.607536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:02.607536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:02.607536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:02.607536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:02.607536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:02.607536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:02.607536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:02.607536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:02.607536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:02.607536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:02.607536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:02.607536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:02.607536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:02.607536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:02.607536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:02.613426 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:55:02.613426 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:55:02.613426 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:02.613426 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:02.613426 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 311/652] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:02.613426 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb82970b-5dc9-4e65-b47b-7210dda4d6eb None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:02.635013 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:02.641252 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b721c9-e856-4dd2-9433-c8644c98b9d5 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:02.678637 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079eeed7-0d74-4031-95ea-3a35a610114a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:02.679304 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079eeed7-0d74-4031-95ea-3a35a610114a None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:02.679591 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079eeed7-0d74-4031-95ea-3a35a610114a None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:02.679901 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079eeed7-0d74-4031-95ea-3a35a610114a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:02.680766 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079eeed7-0d74-4031-95ea-3a35a610114a None admin] RBAC: Adding query filter params (role.id=750bca9c32ac4aa49adcbee30463f1fa, scope.project.id=7938b0346d4240ab89774327d09a89ae, user.id=af26ef96dc9d47dc91549b736939268e) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:02.681093 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079eeed7-0d74-4031-95ea-3a35a610114a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:02.682516 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079eeed7-0d74-4031-95ea-3a35a610114a None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:02.696131 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 342/653] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:55:02 2022] GET /identity/v3/role_assignments?user.id=af26ef96dc9d47dc91549b736939268e&scope.project.id=7938b0346d4240ab89774327d09a89ae&role.id=750bca9c32ac4aa49adcbee30463f1fa => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:02.696131 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079eeed7-0d74-4031-95ea-3a35a610114a None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:02.738658 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb82970b-5dc9-4e65-b47b-7210dda4d6eb None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:02.774717 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:02.780587 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8647e850-80a8-42f3-ab19-2b63360f07c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:02.781426 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647e850-80a8-42f3-ab19-2b63360f07c5 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:02.781911 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647e850-80a8-42f3-ab19-2b63360f07c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:02.782291 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647e850-80a8-42f3-ab19-2b63360f07c5 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:02.783259 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8647e850-80a8-42f3-ab19-2b63360f07c5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:02.783667 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8647e850-80a8-42f3-ab19-2b63360f07c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:02.785320 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8647e850-80a8-42f3-ab19-2b63360f07c5 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:02.801844 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 312/654] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 6 16:55:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:02.802937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8647e850-80a8-42f3-ab19-2b63360f07c5 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:02.835527 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079eeed7-0d74-4031-95ea-3a35a610114a None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:02.892106 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ce532-bf93-4910-a7ff-4f38273bbf40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:02.893189 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ce532-bf93-4910-a7ff-4f38273bbf40 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:02.893623 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ce532-bf93-4910-a7ff-4f38273bbf40 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:02.894047 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8ce532-bf93-4910-a7ff-4f38273bbf40 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:02.904523 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8ce532-bf93-4910-a7ff-4f38273bbf40 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:02.906110 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8ce532-bf93-4910-a7ff-4f38273bbf40 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:02.924541 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-3e8ce532-bf93-4910-a7ff-4f38273bbf40 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 06 16:55:02.924541 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:02.924541 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:02.924541 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:02.924541 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:02.924541 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:02.924541 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:02.924541 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:02.924541 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:02.924541 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:02.924541 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:02.924541 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:02.924541 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 16:55:02.924541 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 16:55:02.924541 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 16:55:02.924541 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 16:55:02.925536 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:02.925536 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:02.925536 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 16:55:02.925536 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:02.925536 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 16:55:02.925536 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:02.925536 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:02.925536 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:02.925536 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:02.925536 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:02.925536 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:02.925536 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:02.925536 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:02.925536 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:02.925536 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:02.925536 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:02.925536 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:02.925536 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:02.926493 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:02.926493 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:02.926493 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 16:55:02.926493 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 16:55:02.926493 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 16:55:02.926493 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 16:55:02.926493 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 16:55:02.926493 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 16:55:02.926493 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 06 16:55:02.926493 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:02.926493 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 343/655] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:55:02 2022] GET /identity/v3/users/glance => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:02.927096 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ce532-bf93-4910-a7ff-4f38273bbf40 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:02.949282 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8647e850-80a8-42f3-ab19-2b63360f07c5 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:02.955065 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:02.999763 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb1278-ed92-44ef-a506-343720b110bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:03.000447 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb1278-ed92-44ef-a506-343720b110bf None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:03.000730 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb1278-ed92-44ef-a506-343720b110bf None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:03.001006 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acb1278-ed92-44ef-a506-343720b110bf None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:03.001740 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acb1278-ed92-44ef-a506-343720b110bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:03.002044 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acb1278-ed92-44ef-a506-343720b110bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:03.003761 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acb1278-ed92-44ef-a506-343720b110bf None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:03.019434 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 313/656] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:55:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:03.020293 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb1278-ed92-44ef-a506-343720b110bf None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:03.060264 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8ce532-bf93-4910-a7ff-4f38273bbf40 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:03.096540 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa6ef2c-5040-474b-8485-1c1cafa04706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:03.097388 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6ef2c-5040-474b-8485-1c1cafa04706 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:03.097669 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6ef2c-5040-474b-8485-1c1cafa04706 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:03.097985 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa6ef2c-5040-474b-8485-1c1cafa04706 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:03.098096 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:03.109335 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa6ef2c-5040-474b-8485-1c1cafa04706 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:03.110765 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa6ef2c-5040-474b-8485-1c1cafa04706 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:03.128964 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-5fa6ef2c-5040-474b-8485-1c1cafa04706 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:03.128964 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:03.128964 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:03.128964 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:03.128964 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:03.128964 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:03.128964 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:03.128964 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:03.128964 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:03.128964 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:03.128964 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:03.128964 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:03.128964 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:55:03.128964 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:55:03.128964 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:55:03.129896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:55:03.129896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:03.129896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:03.129896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:03.129896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:03.129896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:03.129896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:03.129896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:03.129896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:03.129896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:03.129896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:03.129896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:03.129896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:03.129896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:03.129896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:03.129896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:03.129896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:03.130755 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:55:03.130755 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:55:03.130755 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:03.130755 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:03.130755 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 344/657] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:03.131567 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa6ef2c-5040-474b-8485-1c1cafa04706 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:03.153198 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acb1278-ed92-44ef-a506-343720b110bf None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:03.200998 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3a19b-a91f-48ff-a6ec-a7519831bf94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:03.202277 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3a19b-a91f-48ff-a6ec-a7519831bf94 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:03.202575 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3a19b-a91f-48ff-a6ec-a7519831bf94 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:03.203213 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd3a19b-a91f-48ff-a6ec-a7519831bf94 None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:03.204115 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd3a19b-a91f-48ff-a6ec-a7519831bf94 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:03.204671 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd3a19b-a91f-48ff-a6ec-a7519831bf94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:03.206159 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:03.208648 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd3a19b-a91f-48ff-a6ec-a7519831bf94 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:03.227491 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 314/658] 10.222.0.40 () {56 vars in 1137 bytes} [Wed Jul 6 16:55:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:03.228481 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3a19b-a91f-48ff-a6ec-a7519831bf94 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:03.244601 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa6ef2c-5040-474b-8485-1c1cafa04706 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:03.290951 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ad1d5d-0bdf-4766-b734-d800dd33ccd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:03.292489 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad1d5d-0bdf-4766-b734-d800dd33ccd2 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:03.293216 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad1d5d-0bdf-4766-b734-d800dd33ccd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:03.293659 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad1d5d-0bdf-4766-b734-d800dd33ccd2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:03.303018 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ad1d5d-0bdf-4766-b734-d800dd33ccd2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:03.304770 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ad1d5d-0bdf-4766-b734-d800dd33ccd2 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:03.321684 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-e7ad1d5d-0bdf-4766-b734-d800dd33ccd2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:03.321684 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:03.321684 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:03.321684 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:03.321684 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:03.321684 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:03.321684 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:03.321684 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:03.321684 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:03.321684 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:03.321684 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:03.321684 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:03.321684 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 16:55:03.321684 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 16:55:03.321684 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 16:55:03.322827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 16:55:03.322827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:03.322827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:03.322827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:03.322827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:03.322827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:03.322827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:03.322827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:03.322827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:03.322827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:03.322827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:03.322827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:03.322827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:03.322827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:03.322827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:03.322827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:03.322827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:03.323843 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 16:55:03.323843 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 16:55:03.323843 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 16:55:03.323843 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 16:55:03.323843 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 16:55:03.323843 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 16:55:03.323843 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:03.323843 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:03.330203 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 345/659] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:55:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:03.331241 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ad1d5d-0bdf-4766-b734-d800dd33ccd2 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:03.334579 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd3a19b-a91f-48ff-a6ec-a7519831bf94 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:03.353124 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:03.380037 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ffe2f1-7665-4436-be57-b2bdebe0dd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:03.380720 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffe2f1-7665-4436-be57-b2bdebe0dd08 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:03.381007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffe2f1-7665-4436-be57-b2bdebe0dd08 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:03.381283 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ffe2f1-7665-4436-be57-b2bdebe0dd08 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:03.383627 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ffe2f1-7665-4436-be57-b2bdebe0dd08 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:03.384354 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ffe2f1-7665-4436-be57-b2bdebe0dd08 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:03.386044 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ffe2f1-7665-4436-be57-b2bdebe0dd08 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:03.403881 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 315/660] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:55:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:03.405021 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ffe2f1-7665-4436-be57-b2bdebe0dd08 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:03.446265 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ad1d5d-0bdf-4766-b734-d800dd33ccd2 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:03.479095 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:03.485021 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11db4633-1037-467a-9ce0-c3b2d7092b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:03.485911 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db4633-1037-467a-9ce0-c3b2d7092b23 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:03.486385 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db4633-1037-467a-9ce0-c3b2d7092b23 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:03.486857 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11db4633-1037-467a-9ce0-c3b2d7092b23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:03.534877 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ffe2f1-7665-4436-be57-b2bdebe0dd08 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:03.561319 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11db4633-1037-467a-9ce0-c3b2d7092b23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:03.563112 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11db4633-1037-467a-9ce0-c3b2d7092b23 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:03.580144 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f0f10-ed2e-483f-a7f1-390c6941870a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:03.581588 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f0f10-ed2e-483f-a7f1-390c6941870a None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:03.581967 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f0f10-ed2e-483f-a7f1-390c6941870a None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:03.582319 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f0f10-ed2e-483f-a7f1-390c6941870a None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:03.583330 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69f0f10-ed2e-483f-a7f1-390c6941870a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:03.584060 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69f0f10-ed2e-483f-a7f1-390c6941870a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:03.585774 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69f0f10-ed2e-483f-a7f1-390c6941870a None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:03.607790 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 316/661] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 6 16:55:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:03.607790 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f0f10-ed2e-483f-a7f1-390c6941870a None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:03.645345 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:03.692528 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 346/662] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:55:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 16:55:03.692528 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11db4633-1037-467a-9ce0-c3b2d7092b23 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:03.721514 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f0f10-ed2e-483f-a7f1-390c6941870a None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:03.735040 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:03.757049 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb80b0ff-03c9-48ba-b893-8a6ea8963e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:03.757890 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80b0ff-03c9-48ba-b893-8a6ea8963e82 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:03.758208 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80b0ff-03c9-48ba-b893-8a6ea8963e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:03.758488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb80b0ff-03c9-48ba-b893-8a6ea8963e82 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:03.766426 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb80b0ff-03c9-48ba-b893-8a6ea8963e82 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:03.767884 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb80b0ff-03c9-48ba-b893-8a6ea8963e82 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:03.783022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-eb80b0ff-03c9-48ba-b893-8a6ea8963e82 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 06 16:55:03.783022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:03.783022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:03.783022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:03.783022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:03.783022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:03.783022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:03.783022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:03.783022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:03.783022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:03.783022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:03.783022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:03.783022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 16:55:03.783022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 16:55:03.783022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 16:55:03.783022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 16:55:03.784279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:03.784279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:03.784279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 16:55:03.784279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:03.784279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 16:55:03.784279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:03.784279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:03.784279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:03.784279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:03.784279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:03.784279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:03.784279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:03.784279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:03.784279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:03.784279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:03.784279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:03.784279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:03.784279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:03.785612 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:03.785612 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:03.785612 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 16:55:03.785612 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 16:55:03.785612 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 16:55:03.785612 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 16:55:03.785612 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 16:55:03.785612 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 16:55:03.785612 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 06 16:55:03.785612 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:03.785612 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 317/663] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 6 16:55:03 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:03.785612 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c9e25-336d-4c9b-84d9-3627bfc910d5 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:03.786288 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c9e25-336d-4c9b-84d9-3627bfc910d5 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:03.786288 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6c9e25-336d-4c9b-84d9-3627bfc910d5 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:03.787414 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 318/664] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:03.788170 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6c9e25-336d-4c9b-84d9-3627bfc910d5 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:03.811531 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11db4633-1037-467a-9ce0-c3b2d7092b23 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:03.847007 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1f21a-ee79-4aa8-8d81-e7f7e90971b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:03.847659 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1f21a-ee79-4aa8-8d81-e7f7e90971b4 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:03.847977 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1f21a-ee79-4aa8-8d81-e7f7e90971b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:03.848260 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1f21a-ee79-4aa8-8d81-e7f7e90971b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:03.849090 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1f21a-ee79-4aa8-8d81-e7f7e90971b4 None admin] RBAC: Adding query filter params (role.id=750bca9c32ac4aa49adcbee30463f1fa, scope.project.id=7938b0346d4240ab89774327d09a89ae, user.id=a1bfa3e856ae46d48c82195f70aec24f) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:03.849405 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1f21a-ee79-4aa8-8d81-e7f7e90971b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:03.851077 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1f21a-ee79-4aa8-8d81-e7f7e90971b4 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:03.867772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 347/665] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:55:03 2022] GET /identity/v3/role_assignments?user.id=a1bfa3e856ae46d48c82195f70aec24f&scope.project.id=7938b0346d4240ab89774327d09a89ae&role.id=750bca9c32ac4aa49adcbee30463f1fa => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:03.867772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1f21a-ee79-4aa8-8d81-e7f7e90971b4 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:03.909220 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:03.920117 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6c9e25-336d-4c9b-84d9-3627bfc910d5 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:03.966852 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ee54d2-de22-472e-8c5b-110d014b5d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:03.968015 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee54d2-de22-472e-8c5b-110d014b5d0e None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:03.968306 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee54d2-de22-472e-8c5b-110d014b5d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:03.968589 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ee54d2-de22-472e-8c5b-110d014b5d0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:03.974057 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:03.980917 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1f21a-ee79-4aa8-8d81-e7f7e90971b4 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:03.986614 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ee54d2-de22-472e-8c5b-110d014b5d0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:03.989261 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ee54d2-de22-472e-8c5b-110d014b5d0e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:04.013508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-f7ee54d2-de22-472e-8c5b-110d014b5d0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:04.013508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:04.013508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:04.013508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:04.013508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:04.013508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:04.013508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:04.013508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:04.013508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:04.013508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:04.013508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:04.013508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:04.013508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:55:04.013508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:55:04.013508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:55:04.014530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:55:04.014530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:04.014530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:04.014530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:04.014530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:04.014530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:04.014530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:04.014530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:04.014530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:04.014530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:04.014530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:04.014530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:04.014530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:04.014530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:04.014530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:04.014530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:04.014530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:04.015391 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:55:04.015391 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:55:04.015391 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:04.015391 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:04.015391 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 319/666] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:04.018173 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d2577-cefd-4da2-874f-f0b99ae36936 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:04.018173 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d2577-cefd-4da2-874f-f0b99ae36936 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:04.018173 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d2577-cefd-4da2-874f-f0b99ae36936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:04.028305 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcadc917-cfbf-49ae-8fe8-0e278e6d9c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:04.029156 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcadc917-cfbf-49ae-8fe8-0e278e6d9c59 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:04.029509 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcadc917-cfbf-49ae-8fe8-0e278e6d9c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:04.029902 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcadc917-cfbf-49ae-8fe8-0e278e6d9c59 None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:04.030825 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcadc917-cfbf-49ae-8fe8-0e278e6d9c59 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:04.031761 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcadc917-cfbf-49ae-8fe8-0e278e6d9c59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:04.034399 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcadc917-cfbf-49ae-8fe8-0e278e6d9c59 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:04.038249 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-f58d2577-cefd-4da2-874f-f0b99ae36936 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:04.039513 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-f58d2577-cefd-4da2-874f-f0b99ae36936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:04.059747 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 348/667] 10.222.0.40 () {56 vars in 1139 bytes} [Wed Jul 6 16:55:03 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:04.066495 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcadc917-cfbf-49ae-8fe8-0e278e6d9c59 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:04.135826 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 320/668] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:55:04.135826 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d2577-cefd-4da2-874f-f0b99ae36936 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:04.203398 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcadc917-cfbf-49ae-8fe8-0e278e6d9c59 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:04.224051 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:04.249267 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aab007-4a5e-42a6-8d92-ae1b83053d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:04.249947 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aab007-4a5e-42a6-8d92-ae1b83053d27 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:04.250231 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aab007-4a5e-42a6-8d92-ae1b83053d27 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:04.250534 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aab007-4a5e-42a6-8d92-ae1b83053d27 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:04.251053 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d2577-cefd-4da2-874f-f0b99ae36936 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:04.251263 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aab007-4a5e-42a6-8d92-ae1b83053d27 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:04.251568 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aab007-4a5e-42a6-8d92-ae1b83053d27 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:04.253125 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aab007-4a5e-42a6-8d92-ae1b83053d27 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:04.271814 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 349/669] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:55:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:04.272948 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aab007-4a5e-42a6-8d92-ae1b83053d27 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:04.292731 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c77018-7960-4b1a-8e77-0d74c6190eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:04.293763 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c77018-7960-4b1a-8e77-0d74c6190eab None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:04.294045 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:04.294462 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c77018-7960-4b1a-8e77-0d74c6190eab None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:04.294915 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c77018-7960-4b1a-8e77-0d74c6190eab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:04.372569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c77018-7960-4b1a-8e77-0d74c6190eab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:04.374056 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c77018-7960-4b1a-8e77-0d74c6190eab None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:04.407207 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aab007-4a5e-42a6-8d92-ae1b83053d27 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:04.446885 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3321e1d0-eb3f-420b-9932-9bc45d37fa66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:04.447839 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321e1d0-eb3f-420b-9932-9bc45d37fa66 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:04.448133 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321e1d0-eb3f-420b-9932-9bc45d37fa66 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:04.448409 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3321e1d0-eb3f-420b-9932-9bc45d37fa66 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:04.459570 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3321e1d0-eb3f-420b-9932-9bc45d37fa66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:55:04.466365 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:04.467649 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-3321e1d0-eb3f-420b-9932-9bc45d37fa66 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 16:55:04.467649 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:04.467649 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:04.467649 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:04.467649 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:04.467649 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:04.467649 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:04.467649 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:04.467649 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:04.467649 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:04.467649 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:04.467649 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:04.467649 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:55:04.467649 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:55:04.467649 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:55:04.467649 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:55:04.468632 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:55:04.468632 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:55:04.468632 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:55:04.468632 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:55:04.468632 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:04.468632 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:04.468632 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:04.468632 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:04.468632 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:04.468632 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:04.468632 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:04.468632 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:04.468632 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:04.468632 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:04.468632 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:04.468632 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:04.469442 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:04.469442 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:04.469442 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:04.469442 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:04.469442 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:55:04.469442 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:55:04.469442 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:55:04.469442 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:55:04.469442 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:55:04.469442 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:55:04.469442 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 16:55:04.469442 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:04.473923 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 350/670] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 6 16:55:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:04.474690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3321e1d0-eb3f-420b-9932-9bc45d37fa66 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:04.504341 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 321/671] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:55:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 16:55:04.505510 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c77018-7960-4b1a-8e77-0d74c6190eab None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:04.564546 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:04.593784 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3321e1d0-eb3f-420b-9932-9bc45d37fa66 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:04.618191 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c77018-7960-4b1a-8e77-0d74c6190eab None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:04.629459 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17241ea4-c020-4e63-808f-3a9b4f26d2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:04.630318 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17241ea4-c020-4e63-808f-3a9b4f26d2e0 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:04.630608 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17241ea4-c020-4e63-808f-3a9b4f26d2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:04.630888 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17241ea4-c020-4e63-808f-3a9b4f26d2e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:04.638531 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17241ea4-c020-4e63-808f-3a9b4f26d2e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:04.640072 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17241ea4-c020-4e63-808f-3a9b4f26d2e0 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:04.652347 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff03b6-62b7-431d-8941-7c708a8ed439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:04.653015 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff03b6-62b7-431d-8941-7c708a8ed439 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:04.653295 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff03b6-62b7-431d-8941-7c708a8ed439 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:04.653565 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ff03b6-62b7-431d-8941-7c708a8ed439 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:04.654451 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff03b6-62b7-431d-8941-7c708a8ed439 None admin] RBAC: Adding query filter params (name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:04.654757 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff03b6-62b7-431d-8941-7c708a8ed439 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:04.655873 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-17241ea4-c020-4e63-808f-3a9b4f26d2e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:04.655873 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:04.655873 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:04.655873 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:04.655873 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:04.655873 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:04.655873 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:04.655873 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:04.655873 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:04.655873 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:04.655873 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:04.655873 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:04.655873 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 16:55:04.655873 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 16:55:04.655873 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 16:55:04.656717 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 16:55:04.656717 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:04.656717 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:04.656717 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:04.656717 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:04.656717 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:04.656717 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:04.656717 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:04.656717 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:04.656717 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:04.656717 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:04.656717 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:04.656717 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:04.656717 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:04.656717 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:04.656717 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:04.656717 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:04.657635 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 16:55:04.657635 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 16:55:04.657635 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 16:55:04.657635 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 16:55:04.657635 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 16:55:04.657635 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 16:55:04.657635 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:04.657635 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:04.658146 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ff03b6-62b7-431d-8941-7c708a8ed439 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:04.661734 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 351/672] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:55:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:04.662725 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17241ea4-c020-4e63-808f-3a9b4f26d2e0 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:04.665400 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 322/673] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 6 16:55:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:04.666232 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff03b6-62b7-431d-8941-7c708a8ed439 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:04.703828 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:04.792890 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ff03b6-62b7-431d-8941-7c708a8ed439 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:04.795427 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17241ea4-c020-4e63-808f-3a9b4f26d2e0 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:04.834672 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980c0ffd-16e4-462d-b0fc-adc2afcf09ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:04.835657 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c0ffd-16e4-462d-b0fc-adc2afcf09ea None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:04.835923 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910687ac-54a5-4a77-a358-2e0d3562fdaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:04.836429 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c0ffd-16e4-462d-b0fc-adc2afcf09ea None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:04.836884 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980c0ffd-16e4-462d-b0fc-adc2afcf09ea None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:04.837050 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910687ac-54a5-4a77-a358-2e0d3562fdaa None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:04.837445 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910687ac-54a5-4a77-a358-2e0d3562fdaa None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:04.837805 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910687ac-54a5-4a77-a358-2e0d3562fdaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:04.838403 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980c0ffd-16e4-462d-b0fc-adc2afcf09ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:04.838889 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980c0ffd-16e4-462d-b0fc-adc2afcf09ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:04.839880 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:04.840991 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980c0ffd-16e4-462d-b0fc-adc2afcf09ea None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:04.850408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910687ac-54a5-4a77-a358-2e0d3562fdaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:04.852664 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910687ac-54a5-4a77-a358-2e0d3562fdaa None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:04.863886 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 323/674] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 6 16:55:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:04.863886 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980c0ffd-16e4-462d-b0fc-adc2afcf09ea None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:04.885296 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-910687ac-54a5-4a77-a358-2e0d3562fdaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:04.885296 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:04.885296 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:04.885296 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:04.885296 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:04.885296 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:04.885296 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:04.885296 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:04.885296 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:04.885296 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:04.885296 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:04.885296 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:04.885296 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:55:04.885296 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:55:04.885296 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:55:04.886238 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:55:04.886238 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:04.886238 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:04.886238 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:04.886238 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:04.886238 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:04.886238 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:04.886238 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:04.886238 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:04.886238 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:04.886238 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:04.886238 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:04.886238 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:04.886238 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:04.886238 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:04.886238 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:04.886238 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:04.887106 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:55:04.887106 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:55:04.887106 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:04.887106 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:04.889603 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 352/675] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:04.890636 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910687ac-54a5-4a77-a358-2e0d3562fdaa None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:04.971290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:04.977410 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980c0ffd-16e4-462d-b0fc-adc2afcf09ea None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:04.998134 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910687ac-54a5-4a77-a358-2e0d3562fdaa None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:05.016069 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119996e8-d9e8-46b7-99ee-540ad81244bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:05.016873 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119996e8-d9e8-46b7-99ee-540ad81244bc None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:05.017160 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119996e8-d9e8-46b7-99ee-540ad81244bc None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:05.017948 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119996e8-d9e8-46b7-99ee-540ad81244bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:05.029140 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119996e8-d9e8-46b7-99ee-540ad81244bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:05.030591 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119996e8-d9e8-46b7-99ee-540ad81244bc None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:05.033425 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27802f-66c1-4c71-bd97-c99dfbc0a39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:05.034080 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27802f-66c1-4c71-bd97-c99dfbc0a39a None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:05.034367 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27802f-66c1-4c71-bd97-c99dfbc0a39a None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:05.034642 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d27802f-66c1-4c71-bd97-c99dfbc0a39a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:05.035474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d27802f-66c1-4c71-bd97-c99dfbc0a39a None admin] RBAC: Adding query filter params (role.id=750bca9c32ac4aa49adcbee30463f1fa, scope.project.id=7938b0346d4240ab89774327d09a89ae, user.id=635a410f471c43109438d0c39874070b) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:05.035815 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d27802f-66c1-4c71-bd97-c99dfbc0a39a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:05.037408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d27802f-66c1-4c71-bd97-c99dfbc0a39a None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:05.053411 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 353/676] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:55:04 2022] GET /identity/v3/role_assignments?user.id=635a410f471c43109438d0c39874070b&scope.project.id=7938b0346d4240ab89774327d09a89ae&role.id=750bca9c32ac4aa49adcbee30463f1fa => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:05.053411 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27802f-66c1-4c71-bd97-c99dfbc0a39a None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:05.053717 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-119996e8-d9e8-46b7-99ee-540ad81244bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:05.053717 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:05.053717 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:05.053717 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:05.053717 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:05.053717 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:05.053717 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:05.053717 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:05.053717 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:05.053717 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:05.053717 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:05.053717 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:05.053717 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:55:05.053717 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:55:05.053717 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:55:05.054548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:55:05.054548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:05.054548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:05.054548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:05.054548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:05.054548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:05.054548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:05.054548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:05.054548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:05.054548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:05.054548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:05.054548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:05.054548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:05.054548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:05.054548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:05.054548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:05.054548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:05.055481 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:55:05.055481 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:55:05.055481 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:05.055481 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:05.055481 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 324/677] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:05.055481 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74061edc-fecc-422f-8975-9aaff817cb81 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:05.055481 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74061edc-fecc-422f-8975-9aaff817cb81 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:05.055481 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74061edc-fecc-422f-8975-9aaff817cb81 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:05.056586 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 325/678] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:05.057727 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74061edc-fecc-422f-8975-9aaff817cb81 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:05.117644 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:05.177067 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d27802f-66c1-4c71-bd97-c99dfbc0a39a None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:05.205512 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74061edc-fecc-422f-8975-9aaff817cb81 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:05.230167 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212d624f-f2d4-468c-8639-7eedb444ffa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:05.230167 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d624f-f2d4-468c-8639-7eedb444ffa9 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:05.230167 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d624f-f2d4-468c-8639-7eedb444ffa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:05.230167 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d624f-f2d4-468c-8639-7eedb444ffa9 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:05.230568 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212d624f-f2d4-468c-8639-7eedb444ffa9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:05.230568 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212d624f-f2d4-468c-8639-7eedb444ffa9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:05.230568 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212d624f-f2d4-468c-8639-7eedb444ffa9 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:05.249783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 354/679] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:55:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:05.250976 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbafc20-7832-4cf5-9046-e20a47209dad None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:05.258594 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbafc20-7832-4cf5-9046-e20a47209dad None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:05.258594 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbafc20-7832-4cf5-9046-e20a47209dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:05.262388 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed984a83-d39a-4c1d-8cc5-df0a39637a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:05.263500 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed984a83-d39a-4c1d-8cc5-df0a39637a79 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:05.263862 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed984a83-d39a-4c1d-8cc5-df0a39637a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:05.264144 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed984a83-d39a-4c1d-8cc5-df0a39637a79 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:05.264871 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed984a83-d39a-4c1d-8cc5-df0a39637a79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:05.265182 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed984a83-d39a-4c1d-8cc5-df0a39637a79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:05.266564 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed984a83-d39a-4c1d-8cc5-df0a39637a79 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:05.270430 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-efbafc20-7832-4cf5-9046-e20a47209dad None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:05.272755 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-efbafc20-7832-4cf5-9046-e20a47209dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:05.283129 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 326/680] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:55:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:05.283129 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed984a83-d39a-4c1d-8cc5-df0a39637a79 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:05.297772 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:05.359889 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 355/681] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:55:05.360749 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbafc20-7832-4cf5-9046-e20a47209dad None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:05.408216 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:05.424869 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed984a83-d39a-4c1d-8cc5-df0a39637a79 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:05.468856 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbafc20-7832-4cf5-9046-e20a47209dad None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:05.470311 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9317f-df8b-4e60-95c2-f84ed0635d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:05.471984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9317f-df8b-4e60-95c2-f84ed0635d52 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:05.472290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9317f-df8b-4e60-95c2-f84ed0635d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:05.472564 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c9317f-df8b-4e60-95c2-f84ed0635d52 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:05.482762 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c9317f-df8b-4e60-95c2-f84ed0635d52 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:05.484544 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c9317f-df8b-4e60-95c2-f84ed0635d52 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:05.504675 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-c8c9317f-df8b-4e60-95c2-f84ed0635d52 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:05.504675 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:05.504675 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:05.504675 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:05.504675 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:05.504675 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:05.504675 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:05.504675 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:05.504675 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:05.504675 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:05.504675 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:05.504675 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:05.504675 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 16:55:05.504675 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 16:55:05.504675 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 16:55:05.505821 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 16:55:05.505821 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:05.505821 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:05.505821 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:05.505821 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:05.505821 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:05.505821 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:05.505821 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:05.505821 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:05.505821 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:05.505821 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:05.505821 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:05.505821 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:05.505821 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:05.505821 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:05.505821 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:05.505821 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:05.507028 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 16:55:05.507028 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 16:55:05.507028 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 16:55:05.507028 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 16:55:05.507028 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 16:55:05.507028 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 16:55:05.507028 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:05.507028 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:05.507028 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 327/682] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:55:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:05.507655 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a404d2-2e26-41f5-849b-32b0bba64a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:05.507802 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9317f-df8b-4e60-95c2-f84ed0635d52 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:05.509342 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a404d2-2e26-41f5-849b-32b0bba64a3c None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:05.509466 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a404d2-2e26-41f5-849b-32b0bba64a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:05.510719 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a404d2-2e26-41f5-849b-32b0bba64a3c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:05.517822 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a404d2-2e26-41f5-849b-32b0bba64a3c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:05.519336 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a404d2-2e26-41f5-849b-32b0bba64a3c None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:05.535137 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-00a404d2-2e26-41f5-849b-32b0bba64a3c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 06 16:55:05.535137 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:05.535137 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:05.535137 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:05.535137 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:05.535137 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:05.535137 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:05.535137 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:05.535137 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:05.535137 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:05.535137 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:05.535137 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:05.535137 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 16:55:05.535137 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 16:55:05.535137 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 16:55:05.535137 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 16:55:05.536254 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:05.536254 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:05.536254 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 16:55:05.536254 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:05.536254 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 16:55:05.536254 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:05.536254 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:05.536254 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:05.536254 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:05.536254 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:05.536254 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:05.536254 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:05.536254 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:05.536254 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:05.536254 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:05.536254 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:05.536254 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:05.536254 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:05.537749 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:05.537749 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:05.537749 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 16:55:05.537749 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 16:55:05.537749 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 16:55:05.537749 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 16:55:05.537749 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 16:55:05.537749 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 16:55:05.537749 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 06 16:55:05.537749 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:05.537749 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 356/683] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 6 16:55:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:05.537749 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a404d2-2e26-41f5-849b-32b0bba64a3c None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:05.603038 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:05.641152 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:05.645319 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a404d2-2e26-41f5-849b-32b0bba64a3c None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:05.651482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c9317f-df8b-4e60-95c2-f84ed0635d52 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:05.680679 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3dcebb-1abe-4a72-81bb-eb6ab45bae6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:05.681453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3dcebb-1abe-4a72-81bb-eb6ab45bae6d None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:05.681742 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3dcebb-1abe-4a72-81bb-eb6ab45bae6d None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:05.682017 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3dcebb-1abe-4a72-81bb-eb6ab45bae6d None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:05.682809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3dcebb-1abe-4a72-81bb-eb6ab45bae6d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:05.683115 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3dcebb-1abe-4a72-81bb-eb6ab45bae6d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:05.684589 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3dcebb-1abe-4a72-81bb-eb6ab45bae6d None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:05.689230 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26afc3e8-a839-4181-9f25-6916a143df63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:05.690042 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26afc3e8-a839-4181-9f25-6916a143df63 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:05.690325 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26afc3e8-a839-4181-9f25-6916a143df63 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:05.690663 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26afc3e8-a839-4181-9f25-6916a143df63 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:05.697910 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 357/684] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 6 16:55:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:05.698770 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3dcebb-1abe-4a72-81bb-eb6ab45bae6d None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:05.706091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26afc3e8-a839-4181-9f25-6916a143df63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:55:05.722681 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-26afc3e8-a839-4181-9f25-6916a143df63 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:55:05.722681 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:05.722681 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:05.722681 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:05.722681 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:05.722681 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:05.722681 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:05.722681 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:05.722681 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:05.722681 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:05.722681 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:05.722681 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:05.722681 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:55:05.722681 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:55:05.722681 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:55:05.722681 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:55:05.724024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:55:05.724024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:55:05.724024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:55:05.724024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:55:05.724024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:05.724024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:05.724024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:05.724024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:05.724024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:05.724024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:05.724024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:05.724024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:05.724024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:05.724024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:05.724024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:05.724024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:05.725302 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:05.725302 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:05.725302 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:05.725302 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:05.725302 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:55:05.725302 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:55:05.725302 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:55:05.725302 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:55:05.725302 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:55:05.725302 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:55:05.725302 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:55:05.725302 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:05.725302 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 328/685] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 6 16:55:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:05.725302 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26afc3e8-a839-4181-9f25-6916a143df63 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:05.834020 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26afc3e8-a839-4181-9f25-6916a143df63 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:05.834874 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3dcebb-1abe-4a72-81bb-eb6ab45bae6d None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:05.861508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:05.870300 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a5f4ea-1db8-418c-9067-2453efe32d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:05.871023 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5f4ea-1db8-418c-9067-2453efe32d3a None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:05.871323 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5f4ea-1db8-418c-9067-2453efe32d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:05.871579 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a5f4ea-1db8-418c-9067-2453efe32d3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:05.872701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a5f4ea-1db8-418c-9067-2453efe32d3a None admin] RBAC: Adding query filter params (role.id=750bca9c32ac4aa49adcbee30463f1fa, scope.project.id=7938b0346d4240ab89774327d09a89ae, user.id=5f479458a68b4daf9f3a7ae53d6c3763) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:05.872841 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a5f4ea-1db8-418c-9067-2453efe32d3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:05.874316 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a5f4ea-1db8-418c-9067-2453efe32d3a None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:05.880919 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 329/686] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:55:05 2022] GET /identity/v3/role_assignments?user.id=5f479458a68b4daf9f3a7ae53d6c3763&scope.project.id=7938b0346d4240ab89774327d09a89ae&role.id=750bca9c32ac4aa49adcbee30463f1fa => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:05.881831 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a5f4ea-1db8-418c-9067-2453efe32d3a None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:05.882432 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7508040-72f0-40f0-8906-bfee96acaeef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:05.884417 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7508040-72f0-40f0-8906-bfee96acaeef None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:05.884925 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7508040-72f0-40f0-8906-bfee96acaeef None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:05.885573 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7508040-72f0-40f0-8906-bfee96acaeef None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:05.887630 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7508040-72f0-40f0-8906-bfee96acaeef None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:05.888186 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7508040-72f0-40f0-8906-bfee96acaeef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:05.889923 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7508040-72f0-40f0-8906-bfee96acaeef None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:05.912890 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 358/687] 10.222.0.40 () {56 vars in 1135 bytes} [Wed Jul 6 16:55:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:05.920162 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7508040-72f0-40f0-8906-bfee96acaeef None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:05.971460 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:06.061761 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a5f4ea-1db8-418c-9067-2453efe32d3a None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:06.108543 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905be1f8-0f54-40b1-b8ce-3379be503cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:06.109224 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905be1f8-0f54-40b1-b8ce-3379be503cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:06.109510 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905be1f8-0f54-40b1-b8ce-3379be503cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:06.109787 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905be1f8-0f54-40b1-b8ce-3379be503cd9 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:06.110536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905be1f8-0f54-40b1-b8ce-3379be503cd9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:06.110845 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905be1f8-0f54-40b1-b8ce-3379be503cd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:06.112266 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905be1f8-0f54-40b1-b8ce-3379be503cd9 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:06.118837 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7508040-72f0-40f0-8906-bfee96acaeef None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:06.124417 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 330/688] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:55:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:06.129762 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905be1f8-0f54-40b1-b8ce-3379be503cd9 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:06.156427 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d426f4-3ae0-4590-a7c2-1261b67e531a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:06.157478 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d426f4-3ae0-4590-a7c2-1261b67e531a None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:06.157979 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d426f4-3ae0-4590-a7c2-1261b67e531a None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:06.158451 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d426f4-3ae0-4590-a7c2-1261b67e531a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:06.214080 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:06.228979 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d426f4-3ae0-4590-a7c2-1261b67e531a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:06.230715 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d426f4-3ae0-4590-a7c2-1261b67e531a None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:06.236855 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905be1f8-0f54-40b1-b8ce-3379be503cd9 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:06.250315 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:06.274231 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51aca2-ebab-4ce8-8be3-4c19421240ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:06.275031 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51aca2-ebab-4ce8-8be3-4c19421240ae None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:06.275314 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51aca2-ebab-4ce8-8be3-4c19421240ae None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:06.275588 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51aca2-ebab-4ce8-8be3-4c19421240ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:06.286328 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca51aca2-ebab-4ce8-8be3-4c19421240ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:06.287821 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca51aca2-ebab-4ce8-8be3-4c19421240ae None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:06.314395 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-ca51aca2-ebab-4ce8-8be3-4c19421240ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:06.314395 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:06.314395 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:06.314395 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:06.314395 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:06.314395 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:06.314395 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:06.314395 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:06.314395 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:06.314395 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:06.314395 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:06.314395 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:06.314395 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:55:06.314395 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:55:06.314395 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:55:06.315311 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:55:06.315311 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:06.315311 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:06.315311 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:06.315311 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:06.315311 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:06.315311 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:06.315311 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:06.315311 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:06.315311 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:06.315311 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:06.315311 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:06.315311 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:06.315311 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:06.315311 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:06.315311 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:06.315311 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:06.316202 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:55:06.316202 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:55:06.316202 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:06.316202 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:06.316202 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 331/689] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:55:06.316921 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45ccf7-618e-435c-bf31-319477a9a1ba None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:06.317130 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45ccf7-618e-435c-bf31-319477a9a1ba None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:06.317408 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a45ccf7-618e-435c-bf31-319477a9a1ba None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:06.318939 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 332/690] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:06.320818 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a45ccf7-618e-435c-bf31-319477a9a1ba None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:06.342457 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 359/691] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:55:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 16:55:06.344205 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b42d577-5088-4267-99f6-5e8452375859 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:06.344712 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b42d577-5088-4267-99f6-5e8452375859 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:06.345216 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b42d577-5088-4267-99f6-5e8452375859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:06.363070 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-4b42d577-5088-4267-99f6-5e8452375859 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:06.364615 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-4b42d577-5088-4267-99f6-5e8452375859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:06.433809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 360/692] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:55:06.434970 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b42d577-5088-4267-99f6-5e8452375859 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:06.440455 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a45ccf7-618e-435c-bf31-319477a9a1ba None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:06.450711 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:06.481513 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a4e948-b57f-4879-9fba-a31c747a36f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:06.482186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4e948-b57f-4879-9fba-a31c747a36f0 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:06.482474 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4e948-b57f-4879-9fba-a31c747a36f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:06.482772 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a4e948-b57f-4879-9fba-a31c747a36f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:06.483539 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a4e948-b57f-4879-9fba-a31c747a36f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:06.483889 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a4e948-b57f-4879-9fba-a31c747a36f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:06.485365 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a4e948-b57f-4879-9fba-a31c747a36f0 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:06.488853 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:06.499681 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 333/693] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:55:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:06.516288 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a4e948-b57f-4879-9fba-a31c747a36f0 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:06.555550 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b42d577-5088-4267-99f6-5e8452375859 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:06.606461 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e2ebd1-c72d-4c77-b8f0-7dc464f58b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:06.608219 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e2ebd1-c72d-4c77-b8f0-7dc464f58b0f None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:06.608844 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e2ebd1-c72d-4c77-b8f0-7dc464f58b0f None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:06.609393 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e2ebd1-c72d-4c77-b8f0-7dc464f58b0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:06.610953 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a4e948-b57f-4879-9fba-a31c747a36f0 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:06.624103 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e2ebd1-c72d-4c77-b8f0-7dc464f58b0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:06.626425 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e2ebd1-c72d-4c77-b8f0-7dc464f58b0f None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:06.647172 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c92a8-5eaa-4828-8df2-c7438091b39c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:06.648261 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c92a8-5eaa-4828-8df2-c7438091b39c None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:06.648673 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c92a8-5eaa-4828-8df2-c7438091b39c None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:06.649039 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68c92a8-5eaa-4828-8df2-c7438091b39c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:06.649845 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68c92a8-5eaa-4828-8df2-c7438091b39c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:06.651041 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-a5e2ebd1-c72d-4c77-b8f0-7dc464f58b0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:06.651041 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:06.651041 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:06.651041 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:06.651041 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:06.651041 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:06.651041 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:06.651041 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:06.651041 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:06.651041 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:06.651041 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:06.651041 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:06.651041 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:55:06.651041 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:55:06.651041 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:55:06.652098 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:55:06.652098 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:06.652098 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:06.652098 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:06.652098 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:06.652098 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:06.652098 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:06.652098 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:06.652098 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:06.652098 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:06.652098 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:06.652098 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:06.652098 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:06.652098 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:06.652098 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:06.652098 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:06.652098 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:06.653171 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:55:06.653171 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:55:06.653171 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:06.653171 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:06.653467 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68c92a8-5eaa-4828-8df2-c7438091b39c None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:06.655507 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 361/694] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:06.656457 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e2ebd1-c72d-4c77-b8f0-7dc464f58b0f None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:06.668579 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-e68c92a8-5eaa-4828-8df2-c7438091b39c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 06 16:55:06.668579 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:06.668579 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 16:55:06.668579 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 16:55:06.668579 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 16:55:06.668579 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 16:55:06.668579 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 16:55:06.668579 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:06.668579 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 06 16:55:06.668579 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:06.668579 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 16:55:06.668579 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:06.668579 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:06.668579 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:06.668579 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:06.668579 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:06.669775 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:06.669775 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:06.669775 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:06.669775 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:06.669775 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:06.669775 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:06.669775 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:06.669775 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 16:55:06.669775 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 16:55:06.669775 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 16:55:06.669775 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 16:55:06.669775 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:06.669775 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:06.669775 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:06.669775 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:06.669775 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:06.670798 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:06.670798 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:06.670798 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:06.670798 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:06.670798 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:06.670798 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:06.670798 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:06.670798 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:06.670798 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:06.670798 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:06.670798 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:06.670798 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 16:55:06.670798 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:06.670798 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 06 16:55:06.670798 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:06.676954 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 334/695] 10.222.0.40 () {56 vars in 1129 bytes} [Wed Jul 6 16:55:06 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:06.677894 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c92a8-5eaa-4828-8df2-c7438091b39c None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:06.717092 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:06.772026 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:06.775154 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e2ebd1-c72d-4c77-b8f0-7dc464f58b0f None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:06.810518 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68c92a8-5eaa-4828-8df2-c7438091b39c None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:06.850955 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d90e4c7-dc33-47a0-b9f4-f0c81da32291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:06.852500 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d90e4c7-dc33-47a0-b9f4-f0c81da32291 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:06.852888 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d90e4c7-dc33-47a0-b9f4-f0c81da32291 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:06.853272 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d90e4c7-dc33-47a0-b9f4-f0c81da32291 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:06.854114 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d90e4c7-dc33-47a0-b9f4-f0c81da32291 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:06.854540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d90e4c7-dc33-47a0-b9f4-f0c81da32291 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:06.856120 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d90e4c7-dc33-47a0-b9f4-f0c81da32291 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:06.869831 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 335/696] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:55:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:06.870774 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d90e4c7-dc33-47a0-b9f4-f0c81da32291 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:06.928807 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123943fc-3dad-4ac3-892c-e2d9a4337078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:06.929900 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123943fc-3dad-4ac3-892c-e2d9a4337078 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:06.930377 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123943fc-3dad-4ac3-892c-e2d9a4337078 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:06.930824 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123943fc-3dad-4ac3-892c-e2d9a4337078 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:06.938592 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123943fc-3dad-4ac3-892c-e2d9a4337078 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:06.940349 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123943fc-3dad-4ac3-892c-e2d9a4337078 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:06.955868 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-123943fc-3dad-4ac3-892c-e2d9a4337078 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 06 16:55:06.955868 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:06.955868 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:06.955868 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:06.955868 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:06.955868 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:06.955868 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:06.955868 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:06.955868 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:06.955868 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:06.955868 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:06.955868 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:06.955868 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 16:55:06.955868 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 16:55:06.955868 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 16:55:06.955868 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 16:55:06.956848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:06.956848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:06.956848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 16:55:06.956848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:06.956848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 16:55:06.956848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:06.956848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:06.956848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:06.956848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:06.956848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:06.956848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:06.956848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:06.956848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:06.956848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:06.956848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:06.956848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:06.956848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:06.956848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:06.957788 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:06.957788 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:06.957788 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 16:55:06.957788 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 16:55:06.957788 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 16:55:06.957788 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 16:55:06.957788 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 16:55:06.957788 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 16:55:06.957788 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 06 16:55:06.957788 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:06.960436 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 362/697] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Jul 6 16:55:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:06.962031 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123943fc-3dad-4ac3-892c-e2d9a4337078 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:06.985354 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:06.999980 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d90e4c7-dc33-47a0-b9f4-f0c81da32291 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:07.047339 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31753bdd-c38a-4bf7-afef-8f8656435f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:07.048437 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31753bdd-c38a-4bf7-afef-8f8656435f34 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:07.048721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31753bdd-c38a-4bf7-afef-8f8656435f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:07.048995 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31753bdd-c38a-4bf7-afef-8f8656435f34 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:07.049703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31753bdd-c38a-4bf7-afef-8f8656435f34 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:07.050007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31753bdd-c38a-4bf7-afef-8f8656435f34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:07.052161 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31753bdd-c38a-4bf7-afef-8f8656435f34 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:07.064818 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 336/698] 10.222.0.40 () {56 vars in 1124 bytes} [Wed Jul 6 16:55:06 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:07.064818 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31753bdd-c38a-4bf7-afef-8f8656435f34 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:07.076902 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123943fc-3dad-4ac3-892c-e2d9a4337078 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:07.120019 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de28c7-d2fa-4a0c-b71f-d1c92198e55c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:07.120734 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de28c7-d2fa-4a0c-b71f-d1c92198e55c None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:07.121015 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de28c7-d2fa-4a0c-b71f-d1c92198e55c None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:07.121288 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1de28c7-d2fa-4a0c-b71f-d1c92198e55c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:07.130970 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de28c7-d2fa-4a0c-b71f-d1c92198e55c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:07.133290 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1de28c7-d2fa-4a0c-b71f-d1c92198e55c None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:07.149000 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-a1de28c7-d2fa-4a0c-b71f-d1c92198e55c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:07.149000 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:07.149000 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:07.149000 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:07.149000 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:07.149000 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:07.149000 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:07.149000 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:07.149000 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:07.149000 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:07.149000 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:07.149000 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:07.149000 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 16:55:07.149000 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 16:55:07.149000 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 16:55:07.149982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 16:55:07.149982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:07.149982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:07.149982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:07.149982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:07.149982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:07.149982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:07.149982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:07.149982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:07.149982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:07.149982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:07.149982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:07.149982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:07.149982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:07.149982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:07.149982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:07.149982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:07.150864 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 16:55:07.150864 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 16:55:07.150864 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 16:55:07.150864 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 16:55:07.150864 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 16:55:07.150864 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 16:55:07.150864 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:07.150864 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:07.150864 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 363/699] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:55:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:07.151349 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de28c7-d2fa-4a0c-b71f-d1c92198e55c None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:07.174430 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:07.178072 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31753bdd-c38a-4bf7-afef-8f8656435f34 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:07.227765 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d4623-538e-4f28-b1ad-df874925c08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:07.228504 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d4623-538e-4f28-b1ad-df874925c08f None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:07.228790 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d4623-538e-4f28-b1ad-df874925c08f None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:07.229120 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952d4623-538e-4f28-b1ad-df874925c08f None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:07.230062 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952d4623-538e-4f28-b1ad-df874925c08f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:07.230397 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952d4623-538e-4f28-b1ad-df874925c08f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:07.232094 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952d4623-538e-4f28-b1ad-df874925c08f None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:07.256438 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 337/700] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Jul 6 16:55:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:07.257339 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d4623-538e-4f28-b1ad-df874925c08f None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:07.262014 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1de28c7-d2fa-4a0c-b71f-d1c92198e55c None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:07.273437 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:07.300037 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac14f2-fd99-4e6d-b31f-7872a8fb2dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:07.301201 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac14f2-fd99-4e6d-b31f-7872a8fb2dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:07.301445 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac14f2-fd99-4e6d-b31f-7872a8fb2dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:07.301722 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac14f2-fd99-4e6d-b31f-7872a8fb2dd5 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:07.302439 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ac14f2-fd99-4e6d-b31f-7872a8fb2dd5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:07.303103 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ac14f2-fd99-4e6d-b31f-7872a8fb2dd5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:07.305351 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ac14f2-fd99-4e6d-b31f-7872a8fb2dd5 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:07.319882 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 364/701] 10.222.0.40 () {56 vars in 1124 bytes} [Wed Jul 6 16:55:07 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:07.319882 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac14f2-fd99-4e6d-b31f-7872a8fb2dd5 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:07.373357 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952d4623-538e-4f28-b1ad-df874925c08f None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:07.413345 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fe2502-a404-4a57-829c-d0a51929124b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:07.414188 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe2502-a404-4a57-829c-d0a51929124b None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:07.414529 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe2502-a404-4a57-829c-d0a51929124b None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:07.414878 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe2502-a404-4a57-829c-d0a51929124b None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:07.415904 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fe2502-a404-4a57-829c-d0a51929124b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:07.416282 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fe2502-a404-4a57-829c-d0a51929124b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:07.417937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fe2502-a404-4a57-829c-d0a51929124b None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:07.423800 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:07.433019 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 338/702] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 6 16:55:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:07.437057 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650044b-f5c6-4bcc-a973-6fd0a84989a1 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:07.437057 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650044b-f5c6-4bcc-a973-6fd0a84989a1 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:07.437057 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650044b-f5c6-4bcc-a973-6fd0a84989a1 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:07.440713 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 339/703] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:07.441623 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f650044b-f5c6-4bcc-a973-6fd0a84989a1 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:07.447067 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac14f2-fd99-4e6d-b31f-7872a8fb2dd5 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:07.497226 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0814f25-04c8-410f-aebe-f215b675fa60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:07.497904 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0814f25-04c8-410f-aebe-f215b675fa60 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:07.498185 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0814f25-04c8-410f-aebe-f215b675fa60 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:07.498614 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0814f25-04c8-410f-aebe-f215b675fa60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:07.527668 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:07.560618 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f650044b-f5c6-4bcc-a973-6fd0a84989a1 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:07.582706 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0814f25-04c8-410f-aebe-f215b675fa60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:07.584360 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0814f25-04c8-410f-aebe-f215b675fa60 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:07.606207 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29967683-d137-4602-ab28-6d466167f0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:07.607253 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29967683-d137-4602-ab28-6d466167f0a2 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:07.607714 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29967683-d137-4602-ab28-6d466167f0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:07.608234 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29967683-d137-4602-ab28-6d466167f0a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:07.609245 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29967683-d137-4602-ab28-6d466167f0a2 None admin] RBAC: Adding query filter params (role.id=750bca9c32ac4aa49adcbee30463f1fa, scope.project.id=7938b0346d4240ab89774327d09a89ae, user.id=75acacd30e9a4cf2ad58b072b49e3f05) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:07.609834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29967683-d137-4602-ab28-6d466167f0a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:07.611547 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29967683-d137-4602-ab28-6d466167f0a2 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:07.618602 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 340/704] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:55:07 2022] GET /identity/v3/role_assignments?user.id=75acacd30e9a4cf2ad58b072b49e3f05&scope.project.id=7938b0346d4240ab89774327d09a89ae&role.id=750bca9c32ac4aa49adcbee30463f1fa => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:07.620224 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb22b6e2-271b-46ca-b1d7-866e785055d2 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:07.620790 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb22b6e2-271b-46ca-b1d7-866e785055d2 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:07.621288 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb22b6e2-271b-46ca-b1d7-866e785055d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:07.640489 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-cb22b6e2-271b-46ca-b1d7-866e785055d2 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:07.642105 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-cb22b6e2-271b-46ca-b1d7-866e785055d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:07.675632 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:07.710271 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 365/705] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:55:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 16:55:07.716678 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0814f25-04c8-410f-aebe-f215b675fa60 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:07.722275 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 341/706] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:55:07.811430 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb22b6e2-271b-46ca-b1d7-866e785055d2 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:07.867773 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0814f25-04c8-410f-aebe-f215b675fa60 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:07.916089 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:07.920455 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636b2c2-a1ca-4785-821f-32518b800fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:07.921223 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636b2c2-a1ca-4785-821f-32518b800fbb None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:07.922186 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636b2c2-a1ca-4785-821f-32518b800fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:07.922186 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1636b2c2-a1ca-4785-821f-32518b800fbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:07.932818 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1636b2c2-a1ca-4785-821f-32518b800fbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:07.934267 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1636b2c2-a1ca-4785-821f-32518b800fbb None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:07.954015 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-1636b2c2-a1ca-4785-821f-32518b800fbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:07.954015 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:07.954015 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:07.954015 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:07.954015 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:07.954015 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:07.954015 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:07.954015 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:07.954015 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:07.954015 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:07.954015 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:07.954015 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:07.954015 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:55:07.954015 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:55:07.954015 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:55:07.954950 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:55:07.954950 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:07.954950 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:07.954950 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:07.954950 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:07.954950 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:07.954950 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:07.954950 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:07.954950 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:07.954950 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:07.954950 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:07.954950 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:07.954950 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:07.954950 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:07.954950 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:07.954950 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:07.954950 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:07.955902 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:55:07.955902 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:55:07.955902 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:07.955902 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:07.956120 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb22b6e2-271b-46ca-b1d7-866e785055d2 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:07.958000 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 366/707] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:07.963092 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636b2c2-a1ca-4785-821f-32518b800fbb None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:08.003407 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08555b1-d82e-4f32-bba7-b72c79615530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:08.004255 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08555b1-d82e-4f32-bba7-b72c79615530 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:08.004538 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08555b1-d82e-4f32-bba7-b72c79615530 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:08.004809 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08555b1-d82e-4f32-bba7-b72c79615530 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:08.007188 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:08.016955 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08555b1-d82e-4f32-bba7-b72c79615530 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:08.018411 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08555b1-d82e-4f32-bba7-b72c79615530 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:08.041023 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-c08555b1-d82e-4f32-bba7-b72c79615530 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:08.041023 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:08.041023 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:08.041023 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:08.041023 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:08.041023 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:08.041023 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:08.041023 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:08.041023 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:08.041023 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:08.041023 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:08.041023 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:08.041023 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:55:08.041023 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:55:08.041023 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:55:08.041981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:55:08.041981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:08.041981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:08.041981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:08.041981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:08.041981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:08.041981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:08.041981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:08.041981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:08.041981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:08.041981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:08.041981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:08.041981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:08.041981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:08.041981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:08.041981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:08.041981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:08.044046 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:55:08.044046 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:55:08.044046 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:08.044046 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:08.044046 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 342/708] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:55:08.050125 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08555b1-d82e-4f32-bba7-b72c79615530 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:08.100479 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1636b2c2-a1ca-4785-821f-32518b800fbb None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:08.145205 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005629e7-155e-4382-881f-8296b0321bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:08.145880 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005629e7-155e-4382-881f-8296b0321bfd None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:08.146428 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005629e7-155e-4382-881f-8296b0321bfd None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:08.146946 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005629e7-155e-4382-881f-8296b0321bfd None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:08.147787 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005629e7-155e-4382-881f-8296b0321bfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:08.148578 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005629e7-155e-4382-881f-8296b0321bfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:08.150260 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005629e7-155e-4382-881f-8296b0321bfd None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:08.164827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 367/709] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:55:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:08.170752 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005629e7-155e-4382-881f-8296b0321bfd None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:08.177280 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08555b1-d82e-4f32-bba7-b72c79615530 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:08.207024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:08.229673 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a59e95-6fd8-494d-b138-1e414c96a035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:08.230607 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a59e95-6fd8-494d-b138-1e414c96a035 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:08.231355 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a59e95-6fd8-494d-b138-1e414c96a035 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:08.231648 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a59e95-6fd8-494d-b138-1e414c96a035 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:08.232699 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a59e95-6fd8-494d-b138-1e414c96a035 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:08.233012 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a59e95-6fd8-494d-b138-1e414c96a035 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:08.235005 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a59e95-6fd8-494d-b138-1e414c96a035 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:08.252338 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 343/710] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:55:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:08.254549 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a33f7-6b8f-462a-8457-b93b0b35c944 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:08.254841 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a33f7-6b8f-462a-8457-b93b0b35c944 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:08.255116 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2a33f7-6b8f-462a-8457-b93b0b35c944 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:08.256952 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 344/711] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:08.258742 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2a33f7-6b8f-462a-8457-b93b0b35c944 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:08.292067 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:08.299676 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005629e7-155e-4382-881f-8296b0321bfd None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:08.342564 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b85c6a-ce76-4dcc-9f0b-3148bcc02115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:08.343357 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b85c6a-ce76-4dcc-9f0b-3148bcc02115 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:08.343636 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b85c6a-ce76-4dcc-9f0b-3148bcc02115 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:08.343949 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b85c6a-ce76-4dcc-9f0b-3148bcc02115 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:08.354420 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b85c6a-ce76-4dcc-9f0b-3148bcc02115 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:08.356222 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b85c6a-ce76-4dcc-9f0b-3148bcc02115 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:08.373105 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2a33f7-6b8f-462a-8457-b93b0b35c944 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:08.373676 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-53b85c6a-ce76-4dcc-9f0b-3148bcc02115 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:08.373676 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:08.373676 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:08.373676 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:08.373676 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:08.373676 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:08.373676 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:08.373676 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:08.373676 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:08.373676 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:08.373676 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:08.373676 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:08.373676 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 16:55:08.373676 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 16:55:08.373676 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 16:55:08.375547 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 16:55:08.375547 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:08.375547 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:08.375547 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:08.375547 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:08.375547 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:08.375547 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:08.375547 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:08.375547 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:08.375547 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:08.375547 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:08.375547 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:08.375547 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:08.375547 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:08.375547 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:08.375547 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:08.375547 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:08.377376 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 16:55:08.377376 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 16:55:08.377376 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 16:55:08.377376 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 16:55:08.377376 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 16:55:08.377376 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 16:55:08.377376 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:08.377376 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:08.377376 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 368/712] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:55:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:08.377376 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8339fa0-30cd-4079-9329-e2b70dc8bb2a None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:08.378318 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8339fa0-30cd-4079-9329-e2b70dc8bb2a None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:08.378318 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8339fa0-30cd-4079-9329-e2b70dc8bb2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:08.398399 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-c8339fa0-30cd-4079-9329-e2b70dc8bb2a None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:08.400325 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c8339fa0-30cd-4079-9329-e2b70dc8bb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:08.415307 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80568a19-5c75-42ff-9986-891b0effea22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:08.416515 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80568a19-5c75-42ff-9986-891b0effea22 None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:08.416908 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80568a19-5c75-42ff-9986-891b0effea22 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:08.417314 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80568a19-5c75-42ff-9986-891b0effea22 None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:08.418184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80568a19-5c75-42ff-9986-891b0effea22 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:08.419944 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80568a19-5c75-42ff-9986-891b0effea22 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:08.472007 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 369/713] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:55:08.472772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8339fa0-30cd-4079-9329-e2b70dc8bb2a None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:08.526157 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 345/714] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 6 16:55:08 2022] POST /identity/v3/users => generated 254 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 16:55:08.526998 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80568a19-5c75-42ff-9986-891b0effea22 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:08.607314 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:08.612859 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:08.639617 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8339fa0-30cd-4079-9329-e2b70dc8bb2a None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:08.720254 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb048bd-cb07-4c67-a822-85f404123215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:08.720942 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb048bd-cb07-4c67-a822-85f404123215 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:08.721228 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb048bd-cb07-4c67-a822-85f404123215 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:08.721505 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb048bd-cb07-4c67-a822-85f404123215 None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:08.725626 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb048bd-cb07-4c67-a822-85f404123215 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:08.725626 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb048bd-cb07-4c67-a822-85f404123215 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:08.725626 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb048bd-cb07-4c67-a822-85f404123215 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:08.752707 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 370/715] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 6 16:55:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:08.756944 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80568a19-5c75-42ff-9986-891b0effea22 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:08.759512 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb048bd-cb07-4c67-a822-85f404123215 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:08.806914 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da02110c-20a6-4838-857b-fead43f42c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:08.808504 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02110c-20a6-4838-857b-fead43f42c98 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:08.809055 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02110c-20a6-4838-857b-fead43f42c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:08.809531 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da02110c-20a6-4838-857b-fead43f42c98 None admin] PATH_INFO: `/v3/services/network` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:08.810785 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da02110c-20a6-4838-857b-fead43f42c98 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:08.812774 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da02110c-20a6-4838-857b-fead43f42c98 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:08.828095 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-da02110c-20a6-4838-857b-fead43f42c98 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 06 16:55:08.828095 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:08.828095 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 16:55:08.828095 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 16:55:08.828095 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 16:55:08.828095 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 16:55:08.828095 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 16:55:08.828095 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:08.828095 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 06 16:55:08.828095 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:08.828095 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 16:55:08.828095 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:08.828095 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:08.828095 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:08.828095 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:08.828095 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:08.828095 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:08.829083 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:08.829083 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:08.829083 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:08.829083 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:08.829083 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:08.829083 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:08.829083 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 16:55:08.829083 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 16:55:08.829083 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 16:55:08.829083 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 16:55:08.829083 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:08.829083 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:08.829083 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:08.829083 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:08.829083 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:08.829083 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:08.829903 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:08.829903 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:08.829903 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:08.829903 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:08.829903 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:08.829903 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:08.829903 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:08.829903 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:08.829903 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:08.829903 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:08.829903 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 16:55:08.829903 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:08.829903 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 06 16:55:08.829903 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:08.830619 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 346/716] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:55:08 2022] GET /identity/v3/services/network => generated 88 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:08.838001 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da02110c-20a6-4838-857b-fead43f42c98 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:08.882573 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb048bd-cb07-4c67-a822-85f404123215 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:08.920093 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee21c24-5db7-4878-bc0e-6d2fcddd707a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:08.920752 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee21c24-5db7-4878-bc0e-6d2fcddd707a None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:08.921040 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee21c24-5db7-4878-bc0e-6d2fcddd707a None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:08.921320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee21c24-5db7-4878-bc0e-6d2fcddd707a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:08.922155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee21c24-5db7-4878-bc0e-6d2fcddd707a None admin] RBAC: Adding query filter params (role.id=b41f8a7a58e64f749f2099ca45cc04a7, scope.project.id=7938b0346d4240ab89774327d09a89ae, user.id=af26ef96dc9d47dc91549b736939268e) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:08.922467 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee21c24-5db7-4878-bc0e-6d2fcddd707a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:08.923943 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee21c24-5db7-4878-bc0e-6d2fcddd707a None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:08.930356 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 371/717] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:55:08 2022] GET /identity/v3/role_assignments?user.id=af26ef96dc9d47dc91549b736939268e&scope.project.id=7938b0346d4240ab89774327d09a89ae&role.id=b41f8a7a58e64f749f2099ca45cc04a7 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:08.968003 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:08.972623 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da02110c-20a6-4838-857b-fead43f42c98 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:09.015953 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1243c37a-829a-444b-af72-68041839eb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:09.016563 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1243c37a-829a-444b-af72-68041839eb1d None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:09.016851 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1243c37a-829a-444b-af72-68041839eb1d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:09.017130 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1243c37a-829a-444b-af72-68041839eb1d None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:09.017858 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1243c37a-829a-444b-af72-68041839eb1d None admin] RBAC: Adding query filter params (name=network) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:09.018175 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1243c37a-829a-444b-af72-68041839eb1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:09.019579 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1243c37a-829a-444b-af72-68041839eb1d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:09.025764 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 347/718] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Jul 6 16:55:08 2022] GET /identity/v3/services?name=network => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:09.031786 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee21c24-5db7-4878-bc0e-6d2fcddd707a None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:09.046074 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:09.160763 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee21c24-5db7-4878-bc0e-6d2fcddd707a None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:09.197659 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e670aa8c-86a7-4aeb-85a7-d7d740969651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:09.198342 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e670aa8c-86a7-4aeb-85a7-d7d740969651 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:09.198629 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e670aa8c-86a7-4aeb-85a7-d7d740969651 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:09.198909 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e670aa8c-86a7-4aeb-85a7-d7d740969651 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:09.199631 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e670aa8c-86a7-4aeb-85a7-d7d740969651 None admin] RBAC: Adding query filter params (type=network) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:09.199973 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e670aa8c-86a7-4aeb-85a7-d7d740969651 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:09.201346 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e670aa8c-86a7-4aeb-85a7-d7d740969651 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:09.206457 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 372/719] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Jul 6 16:55:09 2022] GET /identity/v3/services?type=network => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:09.601036 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e76b7-48bc-4128-9aec-d04f5f038b77 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:09.601624 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e76b7-48bc-4128-9aec-d04f5f038b77 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:09.602225 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e76b7-48bc-4128-9aec-d04f5f038b77 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:09.603869 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 348/720] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:09.608792 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3593e7ce-ad97-47b2-93ec-0570934e0a56 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:09.609219 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3593e7ce-ad97-47b2-93ec-0570934e0a56 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:09.609387 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3593e7ce-ad97-47b2-93ec-0570934e0a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:09.631587 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-3593e7ce-ad97-47b2-93ec-0570934e0a56 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:09.632858 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-3593e7ce-ad97-47b2-93ec-0570934e0a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:09.747498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 373/721] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 16:55:09.775034 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336e76b7-48bc-4128-9aec-d04f5f038b77 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:09.888841 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336e76b7-48bc-4128-9aec-d04f5f038b77 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:09.930746 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1462ce5a-747c-4ec3-9936-175ebca8e359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:09.931829 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1462ce5a-747c-4ec3-9936-175ebca8e359 None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:09.932315 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1462ce5a-747c-4ec3-9936-175ebca8e359 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:09.932793 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1462ce5a-747c-4ec3-9936-175ebca8e359 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:09.933744 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1462ce5a-747c-4ec3-9936-175ebca8e359 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:09.935333 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1462ce5a-747c-4ec3-9936-175ebca8e359 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:10.063606 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 349/722] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Jul 6 16:55:09 2022] POST /identity/v3/services => generated 251 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 16:55:10.207550 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a993c7-9137-4fab-833e-f640105f5fcb None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:10.207941 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a993c7-9137-4fab-833e-f640105f5fcb None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:10.208145 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a993c7-9137-4fab-833e-f640105f5fcb None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:10.209793 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 374/723] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:10.215963 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d8a59-e19f-459f-a1ca-782b31d6a21a None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:10.216257 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d8a59-e19f-459f-a1ca-782b31d6a21a None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:10.216541 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d8a59-e19f-459f-a1ca-782b31d6a21a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:10.245057 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-946d8a59-e19f-459f-a1ca-782b31d6a21a None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:10.246306 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-946d8a59-e19f-459f-a1ca-782b31d6a21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:10.272886 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:10.350055 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 350/724] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:10 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 16:55:10.373270 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a993c7-9137-4fab-833e-f640105f5fcb None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:10.380473 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:10.483559 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a993c7-9137-4fab-833e-f640105f5fcb None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:10.518332 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de711cfa-19a3-4774-8d5b-203797b35353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:10.519122 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de711cfa-19a3-4774-8d5b-203797b35353 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:10.519413 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de711cfa-19a3-4774-8d5b-203797b35353 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:10.519691 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de711cfa-19a3-4774-8d5b-203797b35353 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:10.533778 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de711cfa-19a3-4774-8d5b-203797b35353 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:55:10.542677 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-de711cfa-19a3-4774-8d5b-203797b35353 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:55:10.542677 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:10.542677 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:10.542677 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:10.542677 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:10.542677 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:10.542677 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:10.542677 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:10.542677 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:10.542677 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:10.542677 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:10.542677 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:10.542677 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:55:10.542677 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:55:10.542677 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:55:10.542677 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:55:10.543574 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:55:10.543574 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:55:10.543574 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:55:10.543574 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:55:10.543574 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:10.543574 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:10.543574 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:10.543574 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:10.543574 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:10.543574 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:10.543574 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:10.543574 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:10.543574 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:10.543574 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:10.543574 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:10.543574 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:10.544458 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:10.544458 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:10.544458 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:10.544458 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:10.544458 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:55:10.544458 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:55:10.544458 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:55:10.544458 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:55:10.544458 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:55:10.544458 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:55:10.544458 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:55:10.544458 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:10.545551 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 375/725] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 6 16:55:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:10.551550 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d8a59-e19f-459f-a1ca-782b31d6a21a None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:10.690470 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d8a59-e19f-459f-a1ca-782b31d6a21a None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:10.740578 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d275177-3060-40ee-88f9-731744decf45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:10.741264 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d275177-3060-40ee-88f9-731744decf45 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:10.741545 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d275177-3060-40ee-88f9-731744decf45 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:10.741818 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d275177-3060-40ee-88f9-731744decf45 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:10.742609 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d275177-3060-40ee-88f9-731744decf45 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:10.742915 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d275177-3060-40ee-88f9-731744decf45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:10.744312 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d275177-3060-40ee-88f9-731744decf45 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:10.757238 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 351/726] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:55:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:10.764793 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de711cfa-19a3-4774-8d5b-203797b35353 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:10.870847 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:10.910794 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de711cfa-19a3-4774-8d5b-203797b35353 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:10.958287 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c13866f-0997-4cf7-b317-3d4f7f0bd611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:10.959094 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13866f-0997-4cf7-b317-3d4f7f0bd611 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:10.959379 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13866f-0997-4cf7-b317-3d4f7f0bd611 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:10.959647 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c13866f-0997-4cf7-b317-3d4f7f0bd611 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:10.977214 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c13866f-0997-4cf7-b317-3d4f7f0bd611 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:10.978658 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c13866f-0997-4cf7-b317-3d4f7f0bd611 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:11.004875 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-9c13866f-0997-4cf7-b317-3d4f7f0bd611 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:11.004875 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:11.004875 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:11.004875 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:11.004875 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:11.004875 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:11.004875 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:11.004875 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:11.004875 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:11.004875 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:11.004875 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:11.004875 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:11.004875 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:55:11.004875 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:55:11.004875 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:55:11.005820 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:55:11.005820 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:11.005820 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:11.005820 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:11.005820 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:11.005820 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:11.005820 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:11.005820 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:11.005820 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:11.005820 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:11.005820 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:11.005820 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:11.005820 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:11.005820 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:11.005820 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:11.005820 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:11.005820 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:11.006704 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:55:11.006704 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:55:11.006704 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:11.006704 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:11.007364 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 376/727] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:11.016863 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d275177-3060-40ee-88f9-731744decf45 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:11.040068 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:11.157866 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d275177-3060-40ee-88f9-731744decf45 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:11.200421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eacf99-2c51-4d39-93bb-7b844edc1d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:11.201500 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eacf99-2c51-4d39-93bb-7b844edc1d4b None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:11.201996 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eacf99-2c51-4d39-93bb-7b844edc1d4b None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:11.202512 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eacf99-2c51-4d39-93bb-7b844edc1d4b None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:11.203468 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eacf99-2c51-4d39-93bb-7b844edc1d4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:11.204021 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eacf99-2c51-4d39-93bb-7b844edc1d4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:11.205623 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87eacf99-2c51-4d39-93bb-7b844edc1d4b None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:11.222745 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 352/728] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:55:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:11.228460 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c13866f-0997-4cf7-b317-3d4f7f0bd611 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:11.403574 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b055c1c-a7a0-4b97-98c7-7c3a554b56cd None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:11.404230 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b055c1c-a7a0-4b97-98c7-7c3a554b56cd None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:11.404744 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b055c1c-a7a0-4b97-98c7-7c3a554b56cd None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:11.406193 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 353/729] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:11.411307 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343112f7-520e-4b87-9b12-5c54c7a3b0cb None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:11.411823 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343112f7-520e-4b87-9b12-5c54c7a3b0cb None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:11.412324 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343112f7-520e-4b87-9b12-5c54c7a3b0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:11.417758 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c13866f-0997-4cf7-b317-3d4f7f0bd611 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:11.438168 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-343112f7-520e-4b87-9b12-5c54c7a3b0cb None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:11.439653 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-343112f7-520e-4b87-9b12-5c54c7a3b0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:11.447266 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:11.484157 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39187bc9-5089-4c52-96fa-e8d004235957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:11.486357 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39187bc9-5089-4c52-96fa-e8d004235957 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:11.486876 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39187bc9-5089-4c52-96fa-e8d004235957 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:11.487371 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39187bc9-5089-4c52-96fa-e8d004235957 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:11.499707 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39187bc9-5089-4c52-96fa-e8d004235957 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:11.501118 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39187bc9-5089-4c52-96fa-e8d004235957 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:11.532780 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-39187bc9-5089-4c52-96fa-e8d004235957 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 06 16:55:11.532780 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:11.532780 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:11.532780 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:11.532780 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:11.532780 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:11.532780 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:11.532780 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:11.532780 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:11.532780 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:11.532780 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:11.532780 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:11.532780 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 16:55:11.532780 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 16:55:11.532780 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 16:55:11.532780 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 16:55:11.533804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:11.533804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:11.533804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 16:55:11.533804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:11.533804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 16:55:11.533804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:11.533804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:11.533804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:11.533804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:11.533804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:11.533804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:11.533804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:11.533804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:11.533804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:11.533804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:11.533804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:11.533804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:11.533804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:11.534789 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:11.534789 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:11.534789 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 16:55:11.534789 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 16:55:11.534789 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 16:55:11.534789 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 16:55:11.534789 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 16:55:11.534789 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 16:55:11.534789 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 06 16:55:11.534789 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:11.534789 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 377/730] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 6 16:55:11 2022] GET /identity/v3/users/swift => generated 83 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:55:11.535947 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0d81b-dd0b-48a5-9582-8c48ca9b9d0e None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:11.535947 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0d81b-dd0b-48a5-9582-8c48ca9b9d0e None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:11.536128 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e0d81b-dd0b-48a5-9582-8c48ca9b9d0e None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:11.537660 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 378/731] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:11.540767 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e0d81b-dd0b-48a5-9582-8c48ca9b9d0e None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:11.556649 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:11.563131 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 354/732] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:11 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 06 16:55:11.570425 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3a2de-1b3a-467b-8175-a256cc073faf None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:11.570425 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3a2de-1b3a-467b-8175-a256cc073faf None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:11.570425 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc3a2de-1b3a-467b-8175-a256cc073faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:11.590060 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-bdc3a2de-1b3a-467b-8175-a256cc073faf None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:11.592998 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-bdc3a2de-1b3a-467b-8175-a256cc073faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:11.683343 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e0d81b-dd0b-48a5-9582-8c48ca9b9d0e None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:11.733440 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ada81-668f-44b2-85c3-3c4415c1eadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:11.734165 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ada81-668f-44b2-85c3-3c4415c1eadc None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:11.734447 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ada81-668f-44b2-85c3-3c4415c1eadc None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:11.734722 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159ada81-668f-44b2-85c3-3c4415c1eadc None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:11.735586 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159ada81-668f-44b2-85c3-3c4415c1eadc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:11.735971 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159ada81-668f-44b2-85c3-3c4415c1eadc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:11.737430 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159ada81-668f-44b2-85c3-3c4415c1eadc None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:11.759000 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 379/733] 10.222.0.40 () {56 vars in 1135 bytes} [Wed Jul 6 16:55:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:11.760000 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ada81-668f-44b2-85c3-3c4415c1eadc None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:11.798576 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:11.817834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 355/734] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:11 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:55:11.819421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc3a2de-1b3a-467b-8175-a256cc073faf None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:11.887835 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159ada81-668f-44b2-85c3-3c4415c1eadc None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:11.929155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9bf1ea-3931-47ca-9c84-bfb0de0abc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:11.930289 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9bf1ea-3931-47ca-9c84-bfb0de0abc8e None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:11.930791 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9bf1ea-3931-47ca-9c84-bfb0de0abc8e None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:11.931254 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9bf1ea-3931-47ca-9c84-bfb0de0abc8e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:11.942661 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9bf1ea-3931-47ca-9c84-bfb0de0abc8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:55:11.949586 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc3a2de-1b3a-467b-8175-a256cc073faf None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:11.951476 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-cb9bf1ea-3931-47ca-9c84-bfb0de0abc8e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 16:55:11.951476 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:11.951476 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:11.951476 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:11.951476 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:11.951476 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:11.951476 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:11.951476 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:11.951476 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:11.951476 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:11.951476 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:11.951476 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:11.951476 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:55:11.951476 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:55:11.951476 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:55:11.951476 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:55:11.952443 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:55:11.952443 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:55:11.952443 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:55:11.952443 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:55:11.952443 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:11.952443 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:11.952443 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:11.952443 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:11.952443 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:11.952443 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:11.952443 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:11.952443 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:11.952443 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:11.952443 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:11.952443 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:11.952443 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:11.953273 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:11.953273 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:11.953273 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:11.953273 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:11.953273 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:55:11.953273 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:55:11.953273 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:55:11.953273 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:55:11.953273 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:55:11.953273 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:55:11.953273 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 16:55:11.953273 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:11.956102 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 380/735] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 6 16:55:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:11.957129 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9bf1ea-3931-47ca-9c84-bfb0de0abc8e None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:11.987416 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:11.993921 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aec34c-8fb8-4851-bccb-a3a449344efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:11.994578 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aec34c-8fb8-4851-bccb-a3a449344efd None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:11.994868 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aec34c-8fb8-4851-bccb-a3a449344efd None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:11.995145 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97aec34c-8fb8-4851-bccb-a3a449344efd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:12.057446 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:12.063481 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9bf1ea-3931-47ca-9c84-bfb0de0abc8e None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:12.066195 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aec34c-8fb8-4851-bccb-a3a449344efd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:12.067659 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97aec34c-8fb8-4851-bccb-a3a449344efd None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:12.102869 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30a84b3-9711-4970-bafc-c104cd5e8414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:12.103661 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a84b3-9711-4970-bafc-c104cd5e8414 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:12.103979 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a84b3-9711-4970-bafc-c104cd5e8414 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:12.104257 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30a84b3-9711-4970-bafc-c104cd5e8414 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:12.115603 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30a84b3-9711-4970-bafc-c104cd5e8414 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:55:12.123293 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-d30a84b3-9711-4970-bafc-c104cd5e8414 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:55:12.123293 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:12.123293 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:12.123293 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:12.123293 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:12.123293 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:12.123293 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:12.123293 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:12.123293 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:12.123293 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:12.123293 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:12.123293 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:12.123293 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:55:12.123293 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:55:12.123293 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:55:12.123293 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:55:12.124442 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:55:12.124442 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:55:12.124442 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:55:12.124442 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:55:12.124442 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:12.124442 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:12.124442 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:12.124442 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:12.124442 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:12.124442 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:12.124442 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:12.124442 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:12.124442 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:12.124442 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:12.124442 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:12.124442 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:12.130157 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:12.130157 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:12.130157 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:12.130157 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:12.130157 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:55:12.130157 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:55:12.130157 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:55:12.130157 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:55:12.130157 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:55:12.130157 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:55:12.130157 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:55:12.130157 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:12.130157 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 381/736] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 6 16:55:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:12.130157 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4fff9-ba4b-44aa-8645-9c3c8c1a8ffe None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:12.130973 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4fff9-ba4b-44aa-8645-9c3c8c1a8ffe None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:12.130973 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b4fff9-ba4b-44aa-8645-9c3c8c1a8ffe None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:12.130973 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 382/737] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:12.130973 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b4fff9-ba4b-44aa-8645-9c3c8c1a8ffe None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:12.183089 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 356/738] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:55:11 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 06 16:55:12.183089 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aec34c-8fb8-4851-bccb-a3a449344efd None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:12.212781 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:12.247403 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b4fff9-ba4b-44aa-8645-9c3c8c1a8ffe None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:12.283981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97aec34c-8fb8-4851-bccb-a3a449344efd None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:12.285410 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1242782-9e1f-48dc-890f-16df7f7e46b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:12.286425 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1242782-9e1f-48dc-890f-16df7f7e46b9 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:12.286708 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1242782-9e1f-48dc-890f-16df7f7e46b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:12.286988 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1242782-9e1f-48dc-890f-16df7f7e46b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:12.287749 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1242782-9e1f-48dc-890f-16df7f7e46b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:12.288107 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1242782-9e1f-48dc-890f-16df7f7e46b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:12.290044 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1242782-9e1f-48dc-890f-16df7f7e46b9 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:12.303028 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 383/739] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 6 16:55:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:12.303028 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0634ebf6-c439-429e-b640-1dbad3e714f1 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:12.303028 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0634ebf6-c439-429e-b640-1dbad3e714f1 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:12.303028 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0634ebf6-c439-429e-b640-1dbad3e714f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:12.317215 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-0634ebf6-c439-429e-b640-1dbad3e714f1 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:12.318373 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-0634ebf6-c439-429e-b640-1dbad3e714f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:12.327523 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74cdafc-b854-42d9-a9d2-2e97cb0d73df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:12.328245 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74cdafc-b854-42d9-a9d2-2e97cb0d73df None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:12.328527 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74cdafc-b854-42d9-a9d2-2e97cb0d73df None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:12.328800 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74cdafc-b854-42d9-a9d2-2e97cb0d73df None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:12.329667 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74cdafc-b854-42d9-a9d2-2e97cb0d73df None admin] RBAC: Adding query filter params (name=admin) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:12.329974 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74cdafc-b854-42d9-a9d2-2e97cb0d73df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:12.331339 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74cdafc-b854-42d9-a9d2-2e97cb0d73df None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:12.332552 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:12.344942 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 357/740] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:55:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:12.344942 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74cdafc-b854-42d9-a9d2-2e97cb0d73df None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:12.401495 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 384/741] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:12 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 06 16:55:12.402331 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0634ebf6-c439-429e-b640-1dbad3e714f1 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:12.445865 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:12.454293 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74cdafc-b854-42d9-a9d2-2e97cb0d73df None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:12.488082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a8bee9-fff8-4adf-b6ae-a1342c1f6e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:12.488812 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a8bee9-fff8-4adf-b6ae-a1342c1f6e27 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:12.489097 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a8bee9-fff8-4adf-b6ae-a1342c1f6e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:12.489378 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a8bee9-fff8-4adf-b6ae-a1342c1f6e27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:12.500021 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a8bee9-fff8-4adf-b6ae-a1342c1f6e27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:12.501466 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a8bee9-fff8-4adf-b6ae-a1342c1f6e27 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:12.518793 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0634ebf6-c439-429e-b640-1dbad3e714f1 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:12.519001 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-b3a8bee9-fff8-4adf-b6ae-a1342c1f6e27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:12.519001 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:12.519001 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:12.519001 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:12.519001 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:12.519001 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:12.519001 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:12.519001 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:12.519001 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:12.519001 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:12.519001 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:12.519001 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:12.519001 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:55:12.519001 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:55:12.519001 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:55:12.519897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:55:12.519897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:12.519897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:12.519897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:12.519897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:12.519897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:12.519897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:12.519897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:12.519897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:12.519897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:12.519897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:12.519897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:12.519897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:12.519897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:12.519897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:12.519897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:12.519897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:12.520777 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:55:12.520777 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:55:12.520777 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:12.520777 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:12.520777 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 358/742] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:12.521065 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a8bee9-fff8-4adf-b6ae-a1342c1f6e27 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:12.559687 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668b7a35-0c41-4143-b45f-b73efbcca626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:12.560824 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b7a35-0c41-4143-b45f-b73efbcca626 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:12.561289 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b7a35-0c41-4143-b45f-b73efbcca626 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:12.561773 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668b7a35-0c41-4143-b45f-b73efbcca626 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:12.572395 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668b7a35-0c41-4143-b45f-b73efbcca626 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:12.573842 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668b7a35-0c41-4143-b45f-b73efbcca626 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:12.581028 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:12.593408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-668b7a35-0c41-4143-b45f-b73efbcca626 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:12.593408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:12.593408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:12.593408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:12.593408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:12.593408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:12.593408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:12.593408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:12.593408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:12.593408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:12.593408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:12.593408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:12.593408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:55:12.593408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:55:12.593408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:55:12.596662 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:55:12.596662 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:12.596662 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:12.596662 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:12.596662 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:12.596662 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:12.596662 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:12.596662 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:12.596662 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:12.596662 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:12.596662 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:12.596662 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:12.596662 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:12.596662 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:12.596662 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:12.596662 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:12.596662 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:12.597537 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:55:12.597537 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:55:12.597537 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:12.597537 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:12.597537 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 385/743] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:12.597537 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668b7a35-0c41-4143-b45f-b73efbcca626 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:12.625485 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a8bee9-fff8-4adf-b6ae-a1342c1f6e27 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:12.658796 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bb0f9b-a45f-4324-8db9-bc5df1aa8166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:12.659590 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb0f9b-a45f-4324-8db9-bc5df1aa8166 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:12.659914 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb0f9b-a45f-4324-8db9-bc5df1aa8166 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:12.660190 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bb0f9b-a45f-4324-8db9-bc5df1aa8166 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:12.670622 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bb0f9b-a45f-4324-8db9-bc5df1aa8166 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:12.672088 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bb0f9b-a45f-4324-8db9-bc5df1aa8166 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:12.674508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:12.689634 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-57bb0f9b-a45f-4324-8db9-bc5df1aa8166 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:12.689634 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:12.689634 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:12.689634 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:12.689634 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:12.689634 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:12.689634 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:12.689634 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:12.689634 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:12.689634 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:12.689634 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:12.689634 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:12.689634 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:55:12.689634 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:55:12.689634 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:55:12.691779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:55:12.691779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:12.691779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:12.691779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:12.691779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:12.691779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:12.691779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:12.691779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:12.691779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:12.691779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:12.691779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:12.691779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:12.691779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:12.691779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:12.691779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:12.691779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:12.691779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:12.692669 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:55:12.692669 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:55:12.692669 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:12.692669 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:12.692669 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 359/744] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:12.692669 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bb0f9b-a45f-4324-8db9-bc5df1aa8166 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:12.699863 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668b7a35-0c41-4143-b45f-b73efbcca626 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:12.733208 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075a2272-1ebe-4a9a-8dc9-b4adeb97903f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:12.733865 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a2272-1ebe-4a9a-8dc9-b4adeb97903f None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:12.734149 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a2272-1ebe-4a9a-8dc9-b4adeb97903f None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:12.734424 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075a2272-1ebe-4a9a-8dc9-b4adeb97903f None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:12.735148 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075a2272-1ebe-4a9a-8dc9-b4adeb97903f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:12.736544 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075a2272-1ebe-4a9a-8dc9-b4adeb97903f None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:12.788945 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bb0f9b-a45f-4324-8db9-bc5df1aa8166 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:12.815681 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 386/745] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Jul 6 16:55:12 2022] POST /identity/v3/services => generated 240 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 16:55:12.817077 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b73e6-ea5d-49f6-818f-1ef239f8af00 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:12.817377 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b73e6-ea5d-49f6-818f-1ef239f8af00 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:12.817652 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1b73e6-ea5d-49f6-818f-1ef239f8af00 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:12.818923 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 387/746] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:12.819649 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b73e6-ea5d-49f6-818f-1ef239f8af00 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:12.832825 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3841a2b-3896-4bb0-b851-624cb421ea00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:12.834277 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3841a2b-3896-4bb0-b851-624cb421ea00 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:12.834686 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3841a2b-3896-4bb0-b851-624cb421ea00 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:12.834985 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3841a2b-3896-4bb0-b851-624cb421ea00 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:12.836690 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3841a2b-3896-4bb0-b851-624cb421ea00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:12.837018 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3841a2b-3896-4bb0-b851-624cb421ea00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:12.838802 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3841a2b-3896-4bb0-b851-624cb421ea00 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:12.853867 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 360/747] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:55:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:12.854532 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3841a2b-3896-4bb0-b851-624cb421ea00 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:12.900161 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:12.915817 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:12.949571 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1b73e6-ea5d-49f6-818f-1ef239f8af00 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:12.965936 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3841a2b-3896-4bb0-b851-624cb421ea00 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:12.993592 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceadaee8-eab9-4c75-a8f4-7d4e9098f204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:12.994176 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceadaee8-eab9-4c75-a8f4-7d4e9098f204 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:12.994464 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceadaee8-eab9-4c75-a8f4-7d4e9098f204 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:12.995170 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceadaee8-eab9-4c75-a8f4-7d4e9098f204 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:12.995479 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceadaee8-eab9-4c75-a8f4-7d4e9098f204 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:12.995836 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceadaee8-eab9-4c75-a8f4-7d4e9098f204 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:12.997204 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceadaee8-eab9-4c75-a8f4-7d4e9098f204 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:13.005920 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611fe231-bb32-428e-9778-b9b7f78841b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:13.009101 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fe231-bb32-428e-9778-b9b7f78841b8 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:13.009101 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fe231-bb32-428e-9778-b9b7f78841b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:13.009101 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611fe231-bb32-428e-9778-b9b7f78841b8 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:13.009101 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611fe231-bb32-428e-9778-b9b7f78841b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:13.009101 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611fe231-bb32-428e-9778-b9b7f78841b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:13.009531 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 388/748] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:55:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:13.010075 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611fe231-bb32-428e-9778-b9b7f78841b8 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:13.015312 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c89fc2f-466c-4574-8c5e-4909f4946990 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:13.015610 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c89fc2f-466c-4574-8c5e-4909f4946990 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:13.015924 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c89fc2f-466c-4574-8c5e-4909f4946990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:13.022743 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 361/749] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:55:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:13.023630 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611fe231-bb32-428e-9778-b9b7f78841b8 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:13.033463 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-6c89fc2f-466c-4574-8c5e-4909f4946990 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:13.034652 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-6c89fc2f-466c-4574-8c5e-4909f4946990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:13.104045 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 389/750] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:55:13.104817 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c89fc2f-466c-4574-8c5e-4909f4946990 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:13.129396 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611fe231-bb32-428e-9778-b9b7f78841b8 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:13.143399 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:13.166102 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:13.166313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce169b4d-14d5-4d36-8e6b-96b862197cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:13.167117 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce169b4d-14d5-4d36-8e6b-96b862197cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:13.167317 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce169b4d-14d5-4d36-8e6b-96b862197cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:13.167594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce169b4d-14d5-4d36-8e6b-96b862197cd9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:13.175151 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce169b4d-14d5-4d36-8e6b-96b862197cd9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:13.177250 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce169b4d-14d5-4d36-8e6b-96b862197cd9 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:13.191872 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-ce169b4d-14d5-4d36-8e6b-96b862197cd9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:13.191872 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:13.191872 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:13.191872 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:13.191872 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:13.191872 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:13.191872 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:13.191872 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:13.191872 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:13.191872 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:13.191872 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:13.191872 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:13.191872 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 16:55:13.191872 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 16:55:13.191872 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 16:55:13.192781 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 16:55:13.192781 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:13.192781 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:13.192781 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:13.192781 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:13.192781 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:13.192781 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:13.192781 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:13.192781 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:13.192781 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:13.192781 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:13.192781 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:13.192781 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:13.192781 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:13.192781 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:13.192781 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:13.192781 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:13.193664 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 16:55:13.193664 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 16:55:13.193664 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 16:55:13.193664 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 16:55:13.193664 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 16:55:13.193664 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 16:55:13.193664 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:13.193664 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:13.196038 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 362/751] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:55:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:13.197408 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce169b4d-14d5-4d36-8e6b-96b862197cd9 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:13.219933 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c89fc2f-466c-4574-8c5e-4909f4946990 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:13.255691 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d926231-3071-43ef-88aa-da49ea9262aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:13.256578 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d926231-3071-43ef-88aa-da49ea9262aa None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:13.256860 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d926231-3071-43ef-88aa-da49ea9262aa None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:13.257136 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d926231-3071-43ef-88aa-da49ea9262aa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:13.264959 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d926231-3071-43ef-88aa-da49ea9262aa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:13.266400 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d926231-3071-43ef-88aa-da49ea9262aa None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:13.284859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-8d926231-3071-43ef-88aa-da49ea9262aa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 06 16:55:13.284859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:13.284859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:13.284859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:13.284859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:13.284859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:13.284859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:13.284859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:13.284859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:13.284859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:13.284859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:13.284859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:13.284859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 16:55:13.284859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 16:55:13.284859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 16:55:13.284859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 16:55:13.285861 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:13.285861 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:13.285861 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 16:55:13.285861 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:13.285861 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 16:55:13.285861 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:13.285861 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:13.285861 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:13.285861 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:13.285861 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:13.285861 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:13.285861 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:13.285861 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:13.285861 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:13.285861 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:13.285861 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:13.285861 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:13.285861 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:13.287548 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:13.287548 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:13.287548 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 16:55:13.287548 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 16:55:13.287548 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 16:55:13.287548 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 16:55:13.287548 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 16:55:13.287548 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 16:55:13.287548 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 06 16:55:13.287548 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:13.287548 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 390/752] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Jul 6 16:55:13 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:13.287548 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d926231-3071-43ef-88aa-da49ea9262aa None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:13.316454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce169b4d-14d5-4d36-8e6b-96b862197cd9 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:13.354827 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710a7959-0fa8-4402-904e-c4909b29af4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:13.355551 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a7959-0fa8-4402-904e-c4909b29af4b None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:13.356211 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a7959-0fa8-4402-904e-c4909b29af4b None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:13.356211 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a7959-0fa8-4402-904e-c4909b29af4b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:13.363883 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710a7959-0fa8-4402-904e-c4909b29af4b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:13.365465 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710a7959-0fa8-4402-904e-c4909b29af4b None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:13.380230 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-710a7959-0fa8-4402-904e-c4909b29af4b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 06 16:55:13.380230 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:13.380230 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:13.380230 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:13.380230 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:13.380230 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:13.380230 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:13.380230 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:13.380230 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:13.380230 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:13.380230 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:13.380230 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:13.380230 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 16:55:13.380230 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 16:55:13.380230 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 16:55:13.380230 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 16:55:13.381376 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:13.381376 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:13.381376 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 16:55:13.381376 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:13.381376 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 16:55:13.381376 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:13.381376 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:13.381376 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:13.381376 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:13.381376 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:13.381376 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:13.381376 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:13.381376 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:13.381376 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:13.381376 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:13.381376 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:13.381376 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:13.381376 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:13.387358 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:13.387358 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:13.387358 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 16:55:13.387358 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 16:55:13.387358 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 16:55:13.387358 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 16:55:13.387358 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 16:55:13.387358 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 16:55:13.387358 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 06 16:55:13.387358 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:13.387358 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 363/753] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Jul 6 16:55:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:13.387358 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710a7959-0fa8-4402-904e-c4909b29af4b None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:13.400639 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d926231-3071-43ef-88aa-da49ea9262aa None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:13.410331 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:13.419089 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:13.436524 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba873f35-a9e9-4c71-883d-880104bef6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:13.437522 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba873f35-a9e9-4c71-883d-880104bef6af None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:13.437801 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba873f35-a9e9-4c71-883d-880104bef6af None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:13.438078 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba873f35-a9e9-4c71-883d-880104bef6af None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:13.438730 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba873f35-a9e9-4c71-883d-880104bef6af None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:13.440182 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba873f35-a9e9-4c71-883d-880104bef6af None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:13.454406 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-ba873f35-a9e9-4c71-883d-880104bef6af None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 06 16:55:13.454406 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:13.454406 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 16:55:13.454406 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 16:55:13.454406 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 16:55:13.454406 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 16:55:13.454406 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 16:55:13.454406 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:13.454406 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 06 16:55:13.454406 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:13.454406 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 16:55:13.454406 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:13.454406 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:13.454406 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:13.454406 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:13.454406 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:13.455576 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:13.455576 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:13.455576 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:13.455576 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:13.455576 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:13.455576 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:13.455576 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:13.455576 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 16:55:13.455576 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 16:55:13.455576 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 16:55:13.455576 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 16:55:13.455576 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:13.455576 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:13.455576 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:13.455576 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:13.455576 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:13.456463 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:13.456463 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:13.456463 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:13.456463 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:13.456463 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:13.456463 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:13.456463 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:13.456463 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:13.456463 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:13.456463 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:13.456463 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:13.456463 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 16:55:13.456463 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:13.456463 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 06 16:55:13.456463 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:13.456463 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 391/754] 10.222.0.40 () {56 vars in 1129 bytes} [Wed Jul 6 16:55:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:13.457390 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba873f35-a9e9-4c71-883d-880104bef6af None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:13.495118 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710a7959-0fa8-4402-904e-c4909b29af4b None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:13.533427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746bbd37-4e2c-4351-85fd-2b3a13abcbe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:13.534365 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746bbd37-4e2c-4351-85fd-2b3a13abcbe4 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:13.534810 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746bbd37-4e2c-4351-85fd-2b3a13abcbe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:13.535240 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746bbd37-4e2c-4351-85fd-2b3a13abcbe4 None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:13.536215 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746bbd37-4e2c-4351-85fd-2b3a13abcbe4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:13.536526 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746bbd37-4e2c-4351-85fd-2b3a13abcbe4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:13.537965 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746bbd37-4e2c-4351-85fd-2b3a13abcbe4 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:13.555432 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 364/755] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 6 16:55:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:13.556397 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746bbd37-4e2c-4351-85fd-2b3a13abcbe4 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:13.569414 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba873f35-a9e9-4c71-883d-880104bef6af None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:13.605546 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb6053-039c-4249-91ad-019f0750f568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:13.607019 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb6053-039c-4249-91ad-019f0750f568 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:13.607715 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb6053-039c-4249-91ad-019f0750f568 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:13.608171 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb6053-039c-4249-91ad-019f0750f568 None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:13.609165 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb6053-039c-4249-91ad-019f0750f568 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:13.609874 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb6053-039c-4249-91ad-019f0750f568 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:13.611588 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb6053-039c-4249-91ad-019f0750f568 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:13.637568 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 392/756] 10.222.0.40 () {56 vars in 1149 bytes} [Wed Jul 6 16:55:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:13.637568 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb6053-039c-4249-91ad-019f0750f568 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:13.652448 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:13.661052 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746bbd37-4e2c-4351-85fd-2b3a13abcbe4 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:13.669780 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:13.698694 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2dd661-703b-4e89-a074-69a82daa2771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:13.699369 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2dd661-703b-4e89-a074-69a82daa2771 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:13.699655 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2dd661-703b-4e89-a074-69a82daa2771 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:13.699962 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2dd661-703b-4e89-a074-69a82daa2771 None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:13.700827 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2dd661-703b-4e89-a074-69a82daa2771 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:13.701137 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2dd661-703b-4e89-a074-69a82daa2771 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:13.702557 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2dd661-703b-4e89-a074-69a82daa2771 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:13.722669 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 365/757] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Jul 6 16:55:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:13.723762 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2dd661-703b-4e89-a074-69a82daa2771 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:13.750534 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb6053-039c-4249-91ad-019f0750f568 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:13.786179 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d367859a-b40f-43fc-872b-3cbff8403550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:13.786739 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367859a-b40f-43fc-872b-3cbff8403550 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:13.787032 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367859a-b40f-43fc-872b-3cbff8403550 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:13.787309 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d367859a-b40f-43fc-872b-3cbff8403550 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:13.788043 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d367859a-b40f-43fc-872b-3cbff8403550 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:13.788351 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d367859a-b40f-43fc-872b-3cbff8403550 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:13.790262 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d367859a-b40f-43fc-872b-3cbff8403550 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:13.795100 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 393/758] 10.222.0.40 () {56 vars in 1124 bytes} [Wed Jul 6 16:55:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:13.795686 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d367859a-b40f-43fc-872b-3cbff8403550 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:13.841774 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2dd661-703b-4e89-a074-69a82daa2771 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:13.876865 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed15ba1-2029-4bf9-9eeb-a0ec3ba09279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:13.877497 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed15ba1-2029-4bf9-9eeb-a0ec3ba09279 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:13.877784 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed15ba1-2029-4bf9-9eeb-a0ec3ba09279 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:13.878117 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed15ba1-2029-4bf9-9eeb-a0ec3ba09279 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:13.878971 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed15ba1-2029-4bf9-9eeb-a0ec3ba09279 None admin] RBAC: Adding query filter params (role.id=b41f8a7a58e64f749f2099ca45cc04a7, scope.project.id=7938b0346d4240ab89774327d09a89ae, user.id=75acacd30e9a4cf2ad58b072b49e3f05) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:13.879294 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed15ba1-2029-4bf9-9eeb-a0ec3ba09279 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:13.880917 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed15ba1-2029-4bf9-9eeb-a0ec3ba09279 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:13.886474 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 366/759] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:55:13 2022] GET /identity/v3/role_assignments?user.id=75acacd30e9a4cf2ad58b072b49e3f05&scope.project.id=7938b0346d4240ab89774327d09a89ae&role.id=b41f8a7a58e64f749f2099ca45cc04a7 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:13.887280 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed15ba1-2029-4bf9-9eeb-a0ec3ba09279 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:13.916762 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:13.917667 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:13.932689 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d367859a-b40f-43fc-872b-3cbff8403550 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:13.980901 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d5b069-db55-4174-a82d-cd916efeae4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:13.982988 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d5b069-db55-4174-a82d-cd916efeae4b None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:13.983611 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d5b069-db55-4174-a82d-cd916efeae4b None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:13.984154 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d5b069-db55-4174-a82d-cd916efeae4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:14.027945 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed15ba1-2029-4bf9-9eeb-a0ec3ba09279 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:14.063943 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f67763-ea2d-4dfe-8941-f9c6c3304d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:14.064740 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f67763-ea2d-4dfe-8941-f9c6c3304d94 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:14.065020 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f67763-ea2d-4dfe-8941-f9c6c3304d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:14.065292 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f67763-ea2d-4dfe-8941-f9c6c3304d94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:14.065594 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d5b069-db55-4174-a82d-cd916efeae4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:14.067597 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d5b069-db55-4174-a82d-cd916efeae4b None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:14.075961 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f67763-ea2d-4dfe-8941-f9c6c3304d94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:14.077522 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f67763-ea2d-4dfe-8941-f9c6c3304d94 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:14.097272 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-02f67763-ea2d-4dfe-8941-f9c6c3304d94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:14.097272 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:14.097272 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:14.097272 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:14.097272 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:14.097272 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:14.097272 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:14.097272 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:14.097272 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:14.097272 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:14.097272 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:14.097272 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:14.097272 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:55:14.097272 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:55:14.097272 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:55:14.098224 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:55:14.098224 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:14.098224 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:14.098224 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:14.098224 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:14.098224 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:14.098224 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:14.098224 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:14.098224 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:14.098224 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:14.098224 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:14.098224 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:14.098224 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:14.098224 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:14.098224 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:14.098224 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:14.098224 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:14.099101 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:55:14.099101 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:55:14.099101 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:14.099101 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:14.103717 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 367/760] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:14.104627 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f67763-ea2d-4dfe-8941-f9c6c3304d94 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:14.173636 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:14.182884 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 394/761] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:55:13 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 16:55:14.187159 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:14.188087 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d5b069-db55-4174-a82d-cd916efeae4b None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:14.229536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f67763-ea2d-4dfe-8941-f9c6c3304d94 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:14.272185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a7c149-d526-490f-a234-572d65961d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:14.273962 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7c149-d526-490f-a234-572d65961d6d None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:14.274429 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7c149-d526-490f-a234-572d65961d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:14.274935 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a7c149-d526-490f-a234-572d65961d6d None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:14.276057 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a7c149-d526-490f-a234-572d65961d6d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:14.276369 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a7c149-d526-490f-a234-572d65961d6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:14.277742 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a7c149-d526-490f-a234-572d65961d6d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:14.285093 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 368/762] 10.222.0.40 () {56 vars in 1124 bytes} [Wed Jul 6 16:55:14 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:14.286393 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a7c149-d526-490f-a234-572d65961d6d None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:14.326897 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d5b069-db55-4174-a82d-cd916efeae4b None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:14.369077 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1b28d1-0fba-4cd4-8035-03d670ab20b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:14.369077 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b28d1-0fba-4cd4-8035-03d670ab20b5 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:14.369077 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b28d1-0fba-4cd4-8035-03d670ab20b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:14.369077 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b28d1-0fba-4cd4-8035-03d670ab20b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:14.369493 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1b28d1-0fba-4cd4-8035-03d670ab20b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:14.369493 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1b28d1-0fba-4cd4-8035-03d670ab20b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:14.369493 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1b28d1-0fba-4cd4-8035-03d670ab20b5 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:14.384067 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 395/763] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:55:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:14.384950 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1b28d1-0fba-4cd4-8035-03d670ab20b5 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:14.401684 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a7c149-d526-490f-a234-572d65961d6d None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:14.435977 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:14.442247 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e70633-21be-43e4-a8dd-6d51ac5bb4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:14.443047 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e70633-21be-43e4-a8dd-6d51ac5bb4f3 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:14.443331 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e70633-21be-43e4-a8dd-6d51ac5bb4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:14.450524 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e70633-21be-43e4-a8dd-6d51ac5bb4f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:14.450654 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:14.456034 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e70633-21be-43e4-a8dd-6d51ac5bb4f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:14.457304 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e70633-21be-43e4-a8dd-6d51ac5bb4f3 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:14.476742 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-44e70633-21be-43e4-a8dd-6d51ac5bb4f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:14.476742 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:14.476742 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:14.476742 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:14.476742 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:14.476742 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:14.476742 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:14.476742 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:14.476742 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:14.476742 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:14.476742 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:14.476742 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:14.476742 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:55:14.476742 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:55:14.476742 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:55:14.480596 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:55:14.480596 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:14.480596 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:14.480596 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:14.480596 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:14.480596 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:14.480596 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:14.480596 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:14.480596 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:14.480596 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:14.480596 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:14.480596 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:14.480596 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:14.480596 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:14.480596 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:14.480596 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:14.480596 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:14.481482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:55:14.481482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:55:14.481482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:14.481482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:14.481482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 369/764] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:14.481482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e70633-21be-43e4-a8dd-6d51ac5bb4f3 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:14.499215 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1b28d1-0fba-4cd4-8035-03d670ab20b5 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:14.535016 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edfc1d4-82b8-4a61-b29f-efa88a1624ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:14.535972 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edfc1d4-82b8-4a61-b29f-efa88a1624ef None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:14.536398 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edfc1d4-82b8-4a61-b29f-efa88a1624ef None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:14.536894 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edfc1d4-82b8-4a61-b29f-efa88a1624ef None admin] PATH_INFO: `/v3/endpoints` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:14.537826 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edfc1d4-82b8-4a61-b29f-efa88a1624ef None admin] RBAC: Adding query filter params (interface=public, service_id=2d2b9608a6274b10a9c79d8cab677ac0, region_id=RegionOne) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:14.538310 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edfc1d4-82b8-4a61-b29f-efa88a1624ef None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:14.539922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edfc1d4-82b8-4a61-b29f-efa88a1624ef None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:14.549430 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 396/765] 10.222.0.40 () {56 vars in 1251 bytes} [Wed Jul 6 16:55:14 2022] GET /identity/v3/endpoints?service_id=2d2b9608a6274b10a9c79d8cab677ac0&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:14.550721 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edfc1d4-82b8-4a61-b29f-efa88a1624ef None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:14.590100 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e70633-21be-43e4-a8dd-6d51ac5bb4f3 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:14.626518 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c413264b-8309-40fe-b90c-8a38f2479306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:14.627374 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c413264b-8309-40fe-b90c-8a38f2479306 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:14.627665 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c413264b-8309-40fe-b90c-8a38f2479306 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:14.628046 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c413264b-8309-40fe-b90c-8a38f2479306 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:14.636344 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c413264b-8309-40fe-b90c-8a38f2479306 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:14.637799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c413264b-8309-40fe-b90c-8a38f2479306 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:14.654321 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-c413264b-8309-40fe-b90c-8a38f2479306 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:14.654321 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:14.654321 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:14.654321 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:14.654321 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:14.654321 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:14.654321 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:14.654321 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:14.654321 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:14.654321 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:14.654321 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:14.654321 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:14.654321 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 16:55:14.654321 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 16:55:14.654321 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 16:55:14.655236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 16:55:14.655236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:14.655236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:14.655236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:14.655236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:14.655236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:14.655236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:14.655236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:14.655236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:14.655236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:14.655236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:14.655236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:14.655236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:14.655236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:14.655236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:14.655236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:14.655236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:14.656809 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 16:55:14.656809 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 16:55:14.656809 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 16:55:14.656809 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 16:55:14.656809 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 16:55:14.656809 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 16:55:14.656809 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:14.656809 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:14.656809 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 370/766] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:55:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:14.660902 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c413264b-8309-40fe-b90c-8a38f2479306 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:14.662559 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edfc1d4-82b8-4a61-b29f-efa88a1624ef None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:14.683286 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:14.700753 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d68553-ffc4-4828-bb75-a5b48ff8631b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:14.701685 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d68553-ffc4-4828-bb75-a5b48ff8631b None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:14.702167 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d68553-ffc4-4828-bb75-a5b48ff8631b None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:14.702731 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d68553-ffc4-4828-bb75-a5b48ff8631b None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:14.703707 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d68553-ffc4-4828-bb75-a5b48ff8631b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:14.704232 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d68553-ffc4-4828-bb75-a5b48ff8631b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:14.705848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d68553-ffc4-4828-bb75-a5b48ff8631b None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:14.706322 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:14.718819 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 397/767] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:55:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:14.720174 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d68553-ffc4-4828-bb75-a5b48ff8631b None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:14.775764 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c413264b-8309-40fe-b90c-8a38f2479306 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:14.812055 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c34fb4-6dac-4036-9ad3-903577d1ccec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:14.812625 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c34fb4-6dac-4036-9ad3-903577d1ccec None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:14.812914 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c34fb4-6dac-4036-9ad3-903577d1ccec None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:14.813192 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c34fb4-6dac-4036-9ad3-903577d1ccec None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:14.813878 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c34fb4-6dac-4036-9ad3-903577d1ccec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:14.815253 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c34fb4-6dac-4036-9ad3-903577d1ccec None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:14.824446 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 371/768] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Jul 6 16:55:14 2022] GET /identity/v3/services => generated 772 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:14.826305 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c34fb4-6dac-4036-9ad3-903577d1ccec None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:14.836296 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d68553-ffc4-4828-bb75-a5b48ff8631b None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:14.883141 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d2ef2-d960-46b9-80ef-41ebb45acda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:14.883874 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d2ef2-d960-46b9-80ef-41ebb45acda1 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:14.884164 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d2ef2-d960-46b9-80ef-41ebb45acda1 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:14.884447 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d2ef2-d960-46b9-80ef-41ebb45acda1 None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:14.885672 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6d2ef2-d960-46b9-80ef-41ebb45acda1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:14.885990 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6d2ef2-d960-46b9-80ef-41ebb45acda1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:14.888018 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6d2ef2-d960-46b9-80ef-41ebb45acda1 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:14.904932 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 398/769] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 6 16:55:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:14.906728 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbdb745-b384-4e2f-b636-b530e56e47f8 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:14.907219 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbdb745-b384-4e2f-b636-b530e56e47f8 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:14.907694 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbdb745-b384-4e2f-b636-b530e56e47f8 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:14.909579 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 399/770] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:14.911117 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbdb745-b384-4e2f-b636-b530e56e47f8 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:14.970130 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c34fb4-6dac-4036-9ad3-903577d1ccec None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:14.976109 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:14.980861 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:15.015567 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb5070d-7a2f-45c4-8273-3603b7d7ed4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:15.016418 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5070d-7a2f-45c4-8273-3603b7d7ed4f None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:15.016805 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5070d-7a2f-45c4-8273-3603b7d7ed4f None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:15.017073 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb5070d-7a2f-45c4-8273-3603b7d7ed4f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:15.025791 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb5070d-7a2f-45c4-8273-3603b7d7ed4f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:15.027243 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb5070d-7a2f-45c4-8273-3603b7d7ed4f None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:15.032472 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbdb745-b384-4e2f-b636-b530e56e47f8 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:15.043278 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-9bb5070d-7a2f-45c4-8273-3603b7d7ed4f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:15.043278 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:15.043278 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:15.043278 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:15.043278 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:15.043278 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:15.043278 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:15.043278 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:15.043278 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:15.043278 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:15.043278 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:15.043278 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:15.043278 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 16:55:15.043278 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 16:55:15.043278 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 16:55:15.044258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 16:55:15.044258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:15.044258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:15.044258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:15.044258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:15.044258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:15.044258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:15.044258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:15.044258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:15.044258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:15.044258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:15.044258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:15.044258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:15.044258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:15.044258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:15.044258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:15.044258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:15.045779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 16:55:15.045779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 16:55:15.045779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 16:55:15.045779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 16:55:15.045779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 16:55:15.045779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 16:55:15.045779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:15.045779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:15.045779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 372/771] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:55:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:15.046265 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb601945-d725-4a62-996d-ede4e31e7f82 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:15.046265 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb601945-d725-4a62-996d-ede4e31e7f82 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:15.046571 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb601945-d725-4a62-996d-ede4e31e7f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:15.071274 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-fb601945-d725-4a62-996d-ede4e31e7f82 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:15.073248 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-fb601945-d725-4a62-996d-ede4e31e7f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:15.075676 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66347a0b-7c58-4a3e-9750-a66782cff8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:15.076713 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66347a0b-7c58-4a3e-9750-a66782cff8ee None admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:15.076999 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66347a0b-7c58-4a3e-9750-a66782cff8ee None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:15.077282 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66347a0b-7c58-4a3e-9750-a66782cff8ee None admin] PATH_INFO: `/v3/projects/7938b0346d4240ab89774327d09a89ae/users/af26ef96dc9d47dc91549b736939268e/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:15.080147 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66347a0b-7c58-4a3e-9750-a66782cff8ee None admin] RBAC: Authorizing `identity:create_grant(project_id=7938b0346d4240ab89774327d09a89ae, user_id=af26ef96dc9d47dc91549b736939268e, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:15.081692 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66347a0b-7c58-4a3e-9750-a66782cff8ee None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:15.150699 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 373/772] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:55:15.151666 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb601945-d725-4a62-996d-ede4e31e7f82 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:15.153346 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 400/773] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 6 16:55:14 2022] PUT /identity/v3/projects/7938b0346d4240ab89774327d09a89ae/users/af26ef96dc9d47dc91549b736939268e/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 16:55:15.180898 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66347a0b-7c58-4a3e-9750-a66782cff8ee None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:15.273745 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:15.280793 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb601945-d725-4a62-996d-ede4e31e7f82 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:15.326925 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d058970-103a-4e21-aeac-66b6ecdac630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:15.328338 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d058970-103a-4e21-aeac-66b6ecdac630 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:15.328871 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d058970-103a-4e21-aeac-66b6ecdac630 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:15.329338 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d058970-103a-4e21-aeac-66b6ecdac630 None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:15.330336 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d058970-103a-4e21-aeac-66b6ecdac630 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:15.330839 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d058970-103a-4e21-aeac-66b6ecdac630 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:15.332716 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d058970-103a-4e21-aeac-66b6ecdac630 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:15.347348 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 374/774] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 6 16:55:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:15.354931 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66347a0b-7c58-4a3e-9750-a66782cff8ee None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:15.355420 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d058970-103a-4e21-aeac-66b6ecdac630 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:15.376157 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:15.396639 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7e143d-3475-402b-b8bc-0e67f72b19d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:15.396639 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e143d-3475-402b-b8bc-0e67f72b19d5 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:15.396639 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e143d-3475-402b-b8bc-0e67f72b19d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:15.396639 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7e143d-3475-402b-b8bc-0e67f72b19d5 None admin] PATH_INFO: `/v3/services/network` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:15.401227 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7e143d-3475-402b-b8bc-0e67f72b19d5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:15.401227 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7e143d-3475-402b-b8bc-0e67f72b19d5 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:15.408514 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-1f7e143d-3475-402b-b8bc-0e67f72b19d5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 06 16:55:15.408514 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:15.408514 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 16:55:15.408514 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 16:55:15.408514 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 16:55:15.408514 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 16:55:15.408514 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 16:55:15.408514 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:15.408514 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 06 16:55:15.408514 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:15.408514 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 16:55:15.408514 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:15.408514 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:15.408514 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:15.408514 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:15.408514 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:15.408514 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:15.409562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:15.409562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:15.409562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:15.409562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:15.409562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:15.409562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:15.409562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 16:55:15.409562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 16:55:15.409562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 16:55:15.409562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 16:55:15.409562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:15.409562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:15.409562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:15.409562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:15.409562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:15.409562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:15.410408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:15.410408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:15.410408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:15.410408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:15.410408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:15.410408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:15.410408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:15.410408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:15.410408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:15.410408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:15.410408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 16:55:15.410408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:15.410408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 06 16:55:15.410408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:15.410408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 401/775] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:55:15 2022] GET /identity/v3/services/network => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:15.415710 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7e143d-3475-402b-b8bc-0e67f72b19d5 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:15.529820 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d058970-103a-4e21-aeac-66b6ecdac630 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:15.543613 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7e143d-3475-402b-b8bc-0e67f72b19d5 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:15.570740 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bcb506-6e13-4843-b6bc-09dd940fcffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:15.571401 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bcb506-6e13-4843-b6bc-09dd940fcffb None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:15.571680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bcb506-6e13-4843-b6bc-09dd940fcffb None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:15.571980 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bcb506-6e13-4843-b6bc-09dd940fcffb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:15.572812 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bcb506-6e13-4843-b6bc-09dd940fcffb None admin] RBAC: Adding query filter params (role.id=750bca9c32ac4aa49adcbee30463f1fa, scope.project.id=7938b0346d4240ab89774327d09a89ae, user.id=1905e6f8c3e24e3cbfefce2827baa370) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:15.573121 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bcb506-6e13-4843-b6bc-09dd940fcffb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:15.574549 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bcb506-6e13-4843-b6bc-09dd940fcffb None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:15.581691 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 375/776] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:55:15 2022] GET /identity/v3/role_assignments?user.id=1905e6f8c3e24e3cbfefce2827baa370&scope.project.id=7938b0346d4240ab89774327d09a89ae&role.id=750bca9c32ac4aa49adcbee30463f1fa => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:15.620831 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f6c700-0dce-4ae2-8d22-5e5f2d646c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:15.621535 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f6c700-0dce-4ae2-8d22-5e5f2d646c30 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:15.621823 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f6c700-0dce-4ae2-8d22-5e5f2d646c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:15.622101 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f6c700-0dce-4ae2-8d22-5e5f2d646c30 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:15.622842 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f6c700-0dce-4ae2-8d22-5e5f2d646c30 None admin] RBAC: Adding query filter params (name=network) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:15.623150 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f6c700-0dce-4ae2-8d22-5e5f2d646c30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:15.624593 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f6c700-0dce-4ae2-8d22-5e5f2d646c30 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:15.637322 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 402/777] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Jul 6 16:55:15 2022] GET /identity/v3/services?name=network => generated 124 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:15.643055 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bcb506-6e13-4843-b6bc-09dd940fcffb None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:15.682316 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:15.805372 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bcb506-6e13-4843-b6bc-09dd940fcffb None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:15.852832 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4947c577-14c5-475c-a87e-cd40376c6d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:15.853503 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947c577-14c5-475c-a87e-cd40376c6d7d None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:15.853790 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947c577-14c5-475c-a87e-cd40376c6d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:15.854067 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4947c577-14c5-475c-a87e-cd40376c6d7d None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:15.854779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4947c577-14c5-475c-a87e-cd40376c6d7d None admin] RBAC: Adding query filter params (type=network) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:15.855089 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4947c577-14c5-475c-a87e-cd40376c6d7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:15.856545 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4947c577-14c5-475c-a87e-cd40376c6d7d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:15.867925 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 376/778] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Jul 6 16:55:15 2022] GET /identity/v3/services?type=network => generated 350 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:15.873560 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f6c700-0dce-4ae2-8d22-5e5f2d646c30 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:15.909155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:16.013365 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f6c700-0dce-4ae2-8d22-5e5f2d646c30 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:16.061481 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc087f-0b39-4c49-9c81-401037e336bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:16.062130 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc087f-0b39-4c49-9c81-401037e336bc None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:16.062422 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc087f-0b39-4c49-9c81-401037e336bc None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:16.062703 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc087f-0b39-4c49-9c81-401037e336bc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:16.063505 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddc087f-0b39-4c49-9c81-401037e336bc None admin] RBAC: Adding query filter params (interface=public, service_id=aaa7acc3ad9140cda8fad9e6a1466be6, region_id=RegionOne) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:16.063845 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddc087f-0b39-4c49-9c81-401037e336bc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:16.065219 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddc087f-0b39-4c49-9c81-401037e336bc None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:16.075778 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 403/779] 10.222.0.40 () {56 vars in 1251 bytes} [Wed Jul 6 16:55:15 2022] GET /identity/v3/endpoints?service_id=aaa7acc3ad9140cda8fad9e6a1466be6&interface=public®ion_id=RegionOne => generated 194 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:16.081425 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4947c577-14c5-475c-a87e-cd40376c6d7d None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:16.187098 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:16.196817 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93f5d7-066f-48b3-94be-71b8eb0330a4 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:16.197160 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93f5d7-066f-48b3-94be-71b8eb0330a4 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:16.197448 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b93f5d7-066f-48b3-94be-71b8eb0330a4 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:16.198816 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 404/780] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 16:55:16.204272 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093941d1-6ace-4c56-b3d0-500c739ca6a3 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:16.204561 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093941d1-6ace-4c56-b3d0-500c739ca6a3 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:16.204845 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093941d1-6ace-4c56-b3d0-500c739ca6a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:16.211010 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4947c577-14c5-475c-a87e-cd40376c6d7d None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:16.244776 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-093941d1-6ace-4c56-b3d0-500c739ca6a3 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:16.245988 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-093941d1-6ace-4c56-b3d0-500c739ca6a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:16.247619 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad19d5-c568-4ae6-b3e6-c2b9fd55a422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:16.248504 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad19d5-c568-4ae6-b3e6-c2b9fd55a422 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:16.248947 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad19d5-c568-4ae6-b3e6-c2b9fd55a422 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:16.249368 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad19d5-c568-4ae6-b3e6-c2b9fd55a422 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:16.250265 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ad19d5-c568-4ae6-b3e6-c2b9fd55a422 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:16.251809 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ad19d5-c568-4ae6-b3e6-c2b9fd55a422 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:16.257654 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 377/781] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Jul 6 16:55:16 2022] GET /identity/v3/services => generated 772 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:16.354592 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 405/782] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:55:16.411758 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad19d5-c568-4ae6-b3e6-c2b9fd55a422 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:16.569982 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad19d5-c568-4ae6-b3e6-c2b9fd55a422 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:16.605243 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a0f45-6c24-43a2-8235-044f0f117235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:16.605976 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a0f45-6c24-43a2-8235-044f0f117235 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:16.606260 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a0f45-6c24-43a2-8235-044f0f117235 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:16.606532 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16a0f45-6c24-43a2-8235-044f0f117235 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:16.617491 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16a0f45-6c24-43a2-8235-044f0f117235 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:55:16.625233 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-e16a0f45-6c24-43a2-8235-044f0f117235 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:55:16.625233 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:16.625233 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:16.625233 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:16.625233 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:16.625233 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:16.625233 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:16.625233 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:16.625233 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:16.625233 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:16.625233 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:16.625233 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:16.625233 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:55:16.625233 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:55:16.625233 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:55:16.625233 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:55:16.626136 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:55:16.626136 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:55:16.626136 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:55:16.626136 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:55:16.626136 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:16.626136 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:16.626136 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:16.626136 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:16.626136 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:16.626136 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:16.626136 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:16.626136 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:16.626136 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:16.626136 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:16.626136 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:16.626136 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:16.626963 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:16.626963 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:16.626963 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:16.626963 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:16.626963 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:55:16.626963 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:55:16.626963 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:55:16.626963 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:55:16.626963 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:55:16.626963 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:55:16.626963 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:55:16.626963 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:16.631353 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 378/783] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 6 16:55:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:16.633576 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093941d1-6ace-4c56-b3d0-500c739ca6a3 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:16.671067 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:16.859628 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093941d1-6ace-4c56-b3d0-500c739ca6a3 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:16.929286 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1ee729-18a3-430e-8f9f-781c98825f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:16.930221 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ee729-18a3-430e-8f9f-781c98825f32 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:16.930664 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ee729-18a3-430e-8f9f-781c98825f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:16.931070 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1ee729-18a3-430e-8f9f-781c98825f32 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:16.931978 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1ee729-18a3-430e-8f9f-781c98825f32 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:16.932420 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1ee729-18a3-430e-8f9f-781c98825f32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:16.933920 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1ee729-18a3-430e-8f9f-781c98825f32 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:16.944210 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 406/784] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:55:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:16.953089 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a0f45-6c24-43a2-8235-044f0f117235 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:16.974445 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:17.152158 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16a0f45-6c24-43a2-8235-044f0f117235 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:17.225053 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f754cf86-43c0-4ca8-9bdb-9108ed3573e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:17.225882 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754cf86-43c0-4ca8-9bdb-9108ed3573e2 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:17.226222 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754cf86-43c0-4ca8-9bdb-9108ed3573e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:17.227003 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f754cf86-43c0-4ca8-9bdb-9108ed3573e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:17.248993 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f754cf86-43c0-4ca8-9bdb-9108ed3573e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:17.250536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f754cf86-43c0-4ca8-9bdb-9108ed3573e2 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:17.279295 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-f754cf86-43c0-4ca8-9bdb-9108ed3573e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:17.279295 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:17.279295 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:17.279295 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:17.279295 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:17.279295 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:17.279295 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:17.279295 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:17.279295 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:17.279295 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:17.279295 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:17.279295 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:17.279295 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:55:17.279295 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:55:17.279295 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:55:17.280446 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:55:17.280446 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:17.280446 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:17.280446 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:17.280446 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:17.280446 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:17.280446 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:17.280446 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:17.280446 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:17.280446 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:17.280446 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:17.280446 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:17.280446 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:17.280446 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:17.280446 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:17.280446 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:17.280446 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:17.281518 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:55:17.281518 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:55:17.281518 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:17.281518 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:17.281518 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 379/785] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:17.286548 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1ee729-18a3-430e-8f9f-781c98825f32 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:17.397051 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:17.416968 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1ee729-18a3-430e-8f9f-781c98825f32 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:17.459134 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8225fd68-e305-4126-9461-f35fb46fdd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:17.460068 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8225fd68-e305-4126-9461-f35fb46fdd00 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:17.460647 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8225fd68-e305-4126-9461-f35fb46fdd00 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:17.461072 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8225fd68-e305-4126-9461-f35fb46fdd00 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:17.461964 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8225fd68-e305-4126-9461-f35fb46fdd00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:17.462414 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8225fd68-e305-4126-9461-f35fb46fdd00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:17.464181 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8225fd68-e305-4126-9461-f35fb46fdd00 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:17.483490 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 407/786] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:55:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:17.489128 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f754cf86-43c0-4ca8-9bdb-9108ed3573e2 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:17.583282 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9090c5-5c2a-45e4-b50b-593c282c5410 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:17.583594 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9090c5-5c2a-45e4-b50b-593c282c5410 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:17.584012 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9090c5-5c2a-45e4-b50b-593c282c5410 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:17.585603 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 408/787] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 16:55:17.592195 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a43b09-4b46-4f66-ba35-1cc0c6273f50 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:17.592485 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a43b09-4b46-4f66-ba35-1cc0c6273f50 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:17.592768 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a43b09-4b46-4f66-ba35-1cc0c6273f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:17.595186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:17.622492 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-61a43b09-4b46-4f66-ba35-1cc0c6273f50 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:17.623690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-61a43b09-4b46-4f66-ba35-1cc0c6273f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:17.673170 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f754cf86-43c0-4ca8-9bdb-9108ed3573e2 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:17.722630 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a12d495-55fe-4821-b845-6dfc36a54da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:17.723454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a12d495-55fe-4821-b845-6dfc36a54da5 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:17.723764 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a12d495-55fe-4821-b845-6dfc36a54da5 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:17.724064 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a12d495-55fe-4821-b845-6dfc36a54da5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:17.735645 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a12d495-55fe-4821-b845-6dfc36a54da5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:17.737409 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a12d495-55fe-4821-b845-6dfc36a54da5 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:17.753737 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-0a12d495-55fe-4821-b845-6dfc36a54da5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 06 16:55:17.753737 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:17.753737 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:17.753737 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:17.753737 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:17.753737 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:17.753737 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:17.753737 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:17.753737 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:17.753737 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:17.753737 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:17.753737 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:17.753737 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 16:55:17.753737 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 16:55:17.753737 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 16:55:17.753737 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 16:55:17.754726 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:17.754726 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:17.754726 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 16:55:17.754726 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:17.754726 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 16:55:17.754726 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:17.754726 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:17.754726 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:17.754726 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:17.754726 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:17.754726 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:17.754726 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:17.754726 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:17.754726 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:17.754726 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:17.754726 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:17.754726 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:17.754726 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:17.755653 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:17.755653 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:17.755653 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 16:55:17.755653 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 16:55:17.755653 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 16:55:17.755653 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 16:55:17.755653 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 16:55:17.755653 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 16:55:17.755653 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 06 16:55:17.755653 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:17.755653 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 380/788] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 6 16:55:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:17.761120 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 409/789] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:17 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 16:55:17.762939 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a12d495-55fe-4821-b845-6dfc36a54da5 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:17.836166 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a43b09-4b46-4f66-ba35-1cc0c6273f50 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:17.928399 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:17.939713 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a12d495-55fe-4821-b845-6dfc36a54da5 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:17.965915 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:17.970381 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a43b09-4b46-4f66-ba35-1cc0c6273f50 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:17.982265 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580fbd84-42af-44ee-8fd9-18de8d8ccd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:17.982947 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580fbd84-42af-44ee-8fd9-18de8d8ccd77 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:17.983596 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580fbd84-42af-44ee-8fd9-18de8d8ccd77 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:17.983912 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580fbd84-42af-44ee-8fd9-18de8d8ccd77 None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:17.985071 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580fbd84-42af-44ee-8fd9-18de8d8ccd77 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:17.985393 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580fbd84-42af-44ee-8fd9-18de8d8ccd77 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:17.987228 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580fbd84-42af-44ee-8fd9-18de8d8ccd77 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:18.009886 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 381/790] 10.222.0.40 () {56 vars in 1135 bytes} [Wed Jul 6 16:55:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:18.011201 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070fe79-942a-4089-8b4b-91a7a41c1ba8 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:18.011497 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070fe79-942a-4089-8b4b-91a7a41c1ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:18.011812 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4070fe79-942a-4089-8b4b-91a7a41c1ba8 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:18.012853 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e608d9-08c1-4070-8783-c3bfaf1485d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:18.013726 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e608d9-08c1-4070-8783-c3bfaf1485d4 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:18.014015 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e608d9-08c1-4070-8783-c3bfaf1485d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:18.014301 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e608d9-08c1-4070-8783-c3bfaf1485d4 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:18.015008 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e608d9-08c1-4070-8783-c3bfaf1485d4 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:18.016558 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e608d9-08c1-4070-8783-c3bfaf1485d4 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:18.017151 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 382/791] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:18.017898 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070fe79-942a-4089-8b4b-91a7a41c1ba8 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:18.031059 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-b7e608d9-08c1-4070-8783-c3bfaf1485d4 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 06 16:55:18.031059 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:18.031059 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 16:55:18.031059 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 16:55:18.031059 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 16:55:18.031059 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 16:55:18.031059 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 16:55:18.031059 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:18.031059 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 06 16:55:18.031059 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:18.031059 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 16:55:18.031059 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:18.031059 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:18.031059 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:18.031059 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:18.031059 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:18.033200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:18.033200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:18.033200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:18.033200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:18.033200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:18.033200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:18.033200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:18.033200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 16:55:18.033200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 16:55:18.033200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 16:55:18.033200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 16:55:18.033200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:18.033200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:18.033200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:18.033200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:18.033200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:18.035227 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:18.035227 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:18.035227 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:18.035227 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:18.035227 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:18.035227 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:18.035227 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:18.035227 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:18.035227 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:18.035227 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:18.035227 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:18.035227 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 16:55:18.035227 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:18.035227 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 06 16:55:18.035227 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:18.043750 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 410/792] 10.222.0.40 () {56 vars in 1129 bytes} [Wed Jul 6 16:55:17 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:55:18.043750 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f0008-5ca7-4285-a0b8-389cf966647e None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:18.043750 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f0008-5ca7-4285-a0b8-389cf966647e None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:18.043750 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f0008-5ca7-4285-a0b8-389cf966647e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:18.056130 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-492f0008-5ca7-4285-a0b8-389cf966647e None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:18.057306 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-492f0008-5ca7-4285-a0b8-389cf966647e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:18.136001 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 411/793] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:55:18.136903 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492f0008-5ca7-4285-a0b8-389cf966647e None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:18.138549 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4070fe79-942a-4089-8b4b-91a7a41c1ba8 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:18.175937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140e00b-975f-4f4e-812c-515f106f9d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:18.179818 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140e00b-975f-4f4e-812c-515f106f9d0f None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:18.179818 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140e00b-975f-4f4e-812c-515f106f9d0f None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:18.179818 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140e00b-975f-4f4e-812c-515f106f9d0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:18.188848 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7140e00b-975f-4f4e-812c-515f106f9d0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:18.190292 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7140e00b-975f-4f4e-812c-515f106f9d0f None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:18.209571 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-7140e00b-975f-4f4e-812c-515f106f9d0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:18.209571 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:18.209571 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:18.209571 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:18.209571 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:18.209571 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:18.209571 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:18.209571 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:18.209571 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:18.209571 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:18.209571 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:18.209571 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:18.209571 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:55:18.209571 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:55:18.209571 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:55:18.212297 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:55:18.212297 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:18.212297 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:18.212297 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:18.212297 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:18.212297 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:18.212297 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:18.212297 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:18.212297 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:18.212297 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:18.212297 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:18.212297 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:18.212297 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:18.212297 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:18.212297 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:18.212297 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:18.212297 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:18.214431 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:55:18.214431 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:55:18.214431 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:18.214431 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:18.214431 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 383/794] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:18.214431 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140e00b-975f-4f4e-812c-515f106f9d0f None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:18.221988 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:18.245404 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:18.258037 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492f0008-5ca7-4285-a0b8-389cf966647e None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:18.296351 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227fcc97-216f-4e1c-91ec-20a592833adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:18.297145 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227fcc97-216f-4e1c-91ec-20a592833adc None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:18.297510 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227fcc97-216f-4e1c-91ec-20a592833adc None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:18.297861 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227fcc97-216f-4e1c-91ec-20a592833adc None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:18.298580 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227fcc97-216f-4e1c-91ec-20a592833adc None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:18.298892 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227fcc97-216f-4e1c-91ec-20a592833adc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:18.300293 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227fcc97-216f-4e1c-91ec-20a592833adc None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:18.309237 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 412/795] 10.222.0.40 () {56 vars in 1124 bytes} [Wed Jul 6 16:55:18 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:18.309237 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227fcc97-216f-4e1c-91ec-20a592833adc None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:18.326127 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140e00b-975f-4f4e-812c-515f106f9d0f None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:18.362612 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b642dc-b9ab-4ef8-9fda-74ca6ddc051a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:18.363463 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b642dc-b9ab-4ef8-9fda-74ca6ddc051a None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:18.363697 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b642dc-b9ab-4ef8-9fda-74ca6ddc051a None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:18.363998 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b642dc-b9ab-4ef8-9fda-74ca6ddc051a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:18.375602 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b642dc-b9ab-4ef8-9fda-74ca6ddc051a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:55:18.383752 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-29b642dc-b9ab-4ef8-9fda-74ca6ddc051a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:55:18.383752 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:18.383752 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:18.383752 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:18.383752 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:18.383752 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:18.383752 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:18.383752 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:18.383752 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:18.383752 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:18.383752 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:18.383752 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:18.383752 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:55:18.383752 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:55:18.383752 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:55:18.383752 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:55:18.384685 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:55:18.384685 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:55:18.384685 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:55:18.384685 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:55:18.384685 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:18.384685 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:18.384685 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:18.384685 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:18.384685 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:18.384685 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:18.384685 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:18.384685 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:18.384685 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:18.384685 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:18.384685 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:18.384685 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:18.390606 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:18.390606 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:18.390606 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:18.390606 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:18.390606 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:55:18.390606 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:55:18.390606 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:55:18.390606 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:55:18.390606 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:55:18.390606 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:55:18.390606 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:55:18.390606 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:18.390606 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 384/796] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 6 16:55:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:18.390606 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a8c1a-84c0-4270-ac37-0e4ad2b65737 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:18.391368 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a8c1a-84c0-4270-ac37-0e4ad2b65737 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:18.391368 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936a8c1a-84c0-4270-ac37-0e4ad2b65737 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:18.391368 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 385/797] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:18.391368 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a8c1a-84c0-4270-ac37-0e4ad2b65737 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:18.419818 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227fcc97-216f-4e1c-91ec-20a592833adc None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:18.448766 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:18.454408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f219f1-2329-443f-a6eb-61d2b21437b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:18.455078 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f219f1-2329-443f-a6eb-61d2b21437b3 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:18.455365 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f219f1-2329-443f-a6eb-61d2b21437b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:18.455649 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f219f1-2329-443f-a6eb-61d2b21437b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:18.456413 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f219f1-2329-443f-a6eb-61d2b21437b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:18.456720 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f219f1-2329-443f-a6eb-61d2b21437b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:18.458086 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f219f1-2329-443f-a6eb-61d2b21437b3 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:18.475470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 413/798] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:55:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:18.475470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f219f1-2329-443f-a6eb-61d2b21437b3 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:18.488618 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:18.500909 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936a8c1a-84c0-4270-ac37-0e4ad2b65737 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:18.539343 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d232627-5f2c-4684-9997-f6d0d8a8cbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:18.540043 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d232627-5f2c-4684-9997-f6d0d8a8cbf9 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:18.540459 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d232627-5f2c-4684-9997-f6d0d8a8cbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:18.540748 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d232627-5f2c-4684-9997-f6d0d8a8cbf9 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:18.541457 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d232627-5f2c-4684-9997-f6d0d8a8cbf9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:18.541770 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d232627-5f2c-4684-9997-f6d0d8a8cbf9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:18.543135 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d232627-5f2c-4684-9997-f6d0d8a8cbf9 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:18.548736 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 386/799] 10.222.0.40 () {56 vars in 1124 bytes} [Wed Jul 6 16:55:18 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:18.550169 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770643c-4648-4839-857e-785bda3756d4 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:18.550460 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770643c-4648-4839-857e-785bda3756d4 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:18.550737 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f770643c-4648-4839-857e-785bda3756d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:18.566794 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-f770643c-4648-4839-857e-785bda3756d4 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:18.568006 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-f770643c-4648-4839-857e-785bda3756d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:18.589858 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f219f1-2329-443f-a6eb-61d2b21437b3 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:18.628334 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59353f0e-9fef-41ed-b614-c50bed0df90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:18.629006 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59353f0e-9fef-41ed-b614-c50bed0df90f None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:18.629291 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59353f0e-9fef-41ed-b614-c50bed0df90f None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:18.629566 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59353f0e-9fef-41ed-b614-c50bed0df90f None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:18.630305 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59353f0e-9fef-41ed-b614-c50bed0df90f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:18.630610 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59353f0e-9fef-41ed-b614-c50bed0df90f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:18.631995 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59353f0e-9fef-41ed-b614-c50bed0df90f None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:18.638582 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 387/800] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:18 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:55:18.645116 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 414/801] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:55:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:18.645116 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59353f0e-9fef-41ed-b614-c50bed0df90f None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:18.649269 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770643c-4648-4839-857e-785bda3756d4 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:18.697156 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:18.736551 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:18.746569 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59353f0e-9fef-41ed-b614-c50bed0df90f None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:18.752740 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f770643c-4648-4839-857e-785bda3756d4 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:18.779278 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893234fb-065b-4ac0-a31b-7aea8bca5f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:18.779968 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893234fb-065b-4ac0-a31b-7aea8bca5f3a None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:18.780251 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893234fb-065b-4ac0-a31b-7aea8bca5f3a None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:18.780526 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893234fb-065b-4ac0-a31b-7aea8bca5f3a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:18.781248 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893234fb-065b-4ac0-a31b-7aea8bca5f3a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:18.782640 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893234fb-065b-4ac0-a31b-7aea8bca5f3a None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:18.786716 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1716628c-12c7-40d4-a600-1a519baa65e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:18.787509 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1716628c-12c7-40d4-a600-1a519baa65e0 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:18.787841 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1716628c-12c7-40d4-a600-1a519baa65e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:18.788122 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1716628c-12c7-40d4-a600-1a519baa65e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:18.795343 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1716628c-12c7-40d4-a600-1a519baa65e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:18.796814 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1716628c-12c7-40d4-a600-1a519baa65e0 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:18.810361 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-1716628c-12c7-40d4-a600-1a519baa65e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:18.810361 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:18.810361 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:18.810361 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:18.810361 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:18.810361 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:18.810361 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:18.810361 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:18.810361 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:18.810361 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:18.810361 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:18.810361 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:18.810361 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 16:55:18.810361 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 16:55:18.810361 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 16:55:18.811528 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 16:55:18.811528 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:18.811528 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:18.811528 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:18.811528 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:18.811528 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:18.811528 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:18.811528 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:18.811528 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:18.811528 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:18.811528 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:18.811528 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:18.811528 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:18.811528 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:18.811528 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:18.811528 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:18.811528 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:18.827258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 16:55:18.827258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 16:55:18.827258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 16:55:18.827258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 16:55:18.827258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 16:55:18.827258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 16:55:18.827258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:18.827258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:18.827258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 388/802] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:55:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:18.827258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1716628c-12c7-40d4-a600-1a519baa65e0 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:18.908028 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1716628c-12c7-40d4-a600-1a519baa65e0 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:18.908779 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 415/803] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 6 16:55:18 2022] POST /identity/v3/endpoints => generated 346 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 16:55:18.909615 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893234fb-065b-4ac0-a31b-7aea8bca5f3a None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:18.958891 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511be242-56b6-46d7-bc4f-16588806e409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:18.959875 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511be242-56b6-46d7-bc4f-16588806e409 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:18.960158 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511be242-56b6-46d7-bc4f-16588806e409 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:18.960467 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511be242-56b6-46d7-bc4f-16588806e409 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:18.971334 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:18.974762 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511be242-56b6-46d7-bc4f-16588806e409 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:18.976520 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511be242-56b6-46d7-bc4f-16588806e409 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:18.995421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-511be242-56b6-46d7-bc4f-16588806e409 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:18.995421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:18.995421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:18.995421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:18.995421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:18.995421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:18.995421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:18.995421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:18.995421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:18.995421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:18.995421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:18.995421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:18.995421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:55:18.995421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:55:18.995421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:55:18.996586 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:55:18.996586 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:18.996586 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:18.996586 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:18.996586 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:18.996586 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:18.996586 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:18.996586 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:18.996586 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:18.996586 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:18.996586 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:18.996586 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:18.996586 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:18.996586 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:18.996586 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:18.996586 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:18.996586 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:18.997476 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:55:18.997476 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:55:18.997476 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:18.997476 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:18.997476 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 389/804] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:18.998177 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8a160-6340-45cb-bf1f-ff9a4466b059 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:18.998177 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8a160-6340-45cb-bf1f-ff9a4466b059 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:18.998427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e8a160-6340-45cb-bf1f-ff9a4466b059 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:19.003803 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 390/805] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:19.004579 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e8a160-6340-45cb-bf1f-ff9a4466b059 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:19.028158 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893234fb-065b-4ac0-a31b-7aea8bca5f3a None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:19.038508 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:19.064934 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d203f366-ee85-4e6c-b117-c7147f8c2428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:19.065821 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d203f366-ee85-4e6c-b117-c7147f8c2428 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:19.066165 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d203f366-ee85-4e6c-b117-c7147f8c2428 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:19.066445 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d203f366-ee85-4e6c-b117-c7147f8c2428 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:19.078945 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d203f366-ee85-4e6c-b117-c7147f8c2428 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:55:19.087276 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-d203f366-ee85-4e6c-b117-c7147f8c2428 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 16:55:19.087276 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:19.087276 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:19.087276 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:19.087276 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:19.087276 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:19.087276 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:19.087276 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:19.087276 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:19.087276 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:19.087276 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:19.087276 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:19.087276 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:55:19.087276 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:55:19.087276 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:55:19.087276 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:55:19.088353 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:55:19.088353 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:55:19.088353 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:55:19.088353 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:55:19.088353 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:19.088353 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:19.088353 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:19.088353 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:19.088353 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:19.088353 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:19.088353 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:19.088353 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:19.088353 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:19.088353 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:19.088353 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:19.088353 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:19.089317 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:19.089317 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:19.089317 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:19.089317 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:19.089317 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:55:19.089317 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:55:19.089317 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:55:19.089317 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:55:19.089317 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:55:19.089317 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:55:19.089317 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 16:55:19.089317 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:19.089317 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 416/806] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 6 16:55:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:19.091903 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d203f366-ee85-4e6c-b117-c7147f8c2428 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:19.120256 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e8a160-6340-45cb-bf1f-ff9a4466b059 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:19.158532 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851e3d1f-7d92-4840-ac8c-f819bb4ef3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:19.159197 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851e3d1f-7d92-4840-ac8c-f819bb4ef3ca None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:19.159596 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851e3d1f-7d92-4840-ac8c-f819bb4ef3ca None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:19.159905 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851e3d1f-7d92-4840-ac8c-f819bb4ef3ca None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:19.160815 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851e3d1f-7d92-4840-ac8c-f819bb4ef3ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:19.161175 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851e3d1f-7d92-4840-ac8c-f819bb4ef3ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:19.162598 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851e3d1f-7d92-4840-ac8c-f819bb4ef3ca None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:19.180333 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 391/807] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 6 16:55:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:19.180333 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3fb22a-cc6e-4247-bde6-f97ed66769ea None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:19.180333 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3fb22a-cc6e-4247-bde6-f97ed66769ea None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:19.180333 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3fb22a-cc6e-4247-bde6-f97ed66769ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:19.194622 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-9f3fb22a-cc6e-4247-bde6-f97ed66769ea None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:19.196485 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-9f3fb22a-cc6e-4247-bde6-f97ed66769ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:19.200242 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d203f366-ee85-4e6c-b117-c7147f8c2428 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:19.237904 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80cb95b-3446-491a-abd4-7d44447dafbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:19.238575 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80cb95b-3446-491a-abd4-7d44447dafbe None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:19.238860 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80cb95b-3446-491a-abd4-7d44447dafbe None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:19.239141 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80cb95b-3446-491a-abd4-7d44447dafbe None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:19.239899 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80cb95b-3446-491a-abd4-7d44447dafbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:19.240208 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80cb95b-3446-491a-abd4-7d44447dafbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:19.241579 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80cb95b-3446-491a-abd4-7d44447dafbe None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:19.247340 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:19.260227 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 417/808] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:55:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:19.260227 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80cb95b-3446-491a-abd4-7d44447dafbe None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:19.284008 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 392/809] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:55:19.284753 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3fb22a-cc6e-4247-bde6-f97ed66769ea None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:19.286572 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:19.368776 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80cb95b-3446-491a-abd4-7d44447dafbe None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:19.383030 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3fb22a-cc6e-4247-bde6-f97ed66769ea None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:19.408279 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d7c38-2321-4af7-9c4c-ebd1983f04ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:19.408956 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d7c38-2321-4af7-9c4c-ebd1983f04ce None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:19.409236 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d7c38-2321-4af7-9c4c-ebd1983f04ce None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:19.409505 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d7c38-2321-4af7-9c4c-ebd1983f04ce None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:19.410254 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d7c38-2321-4af7-9c4c-ebd1983f04ce None admin] RBAC: Adding query filter params (name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:19.410560 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d7c38-2321-4af7-9c4c-ebd1983f04ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:19.411944 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d7c38-2321-4af7-9c4c-ebd1983f04ce None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:19.418805 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea44a2e2-07b3-46fb-a47d-8901cbb5347c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:19.419708 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea44a2e2-07b3-46fb-a47d-8901cbb5347c None admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:19.420023 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea44a2e2-07b3-46fb-a47d-8901cbb5347c None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:19.424862 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea44a2e2-07b3-46fb-a47d-8901cbb5347c None admin] PATH_INFO: `/v3/projects/7938b0346d4240ab89774327d09a89ae/users/75acacd30e9a4cf2ad58b072b49e3f05/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:19.424986 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 418/810] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 6 16:55:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:19.424986 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d7c38-2321-4af7-9c4c-ebd1983f04ce None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:19.426263 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea44a2e2-07b3-46fb-a47d-8901cbb5347c None admin] RBAC: Authorizing `identity:create_grant(project_id=7938b0346d4240ab89774327d09a89ae, user_id=75acacd30e9a4cf2ad58b072b49e3f05, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:19.427694 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea44a2e2-07b3-46fb-a47d-8901cbb5347c None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:19.515842 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 393/811] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 6 16:55:19 2022] PUT /identity/v3/projects/7938b0346d4240ab89774327d09a89ae/users/75acacd30e9a4cf2ad58b072b49e3f05/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 16:55:19.515842 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea44a2e2-07b3-46fb-a47d-8901cbb5347c None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:19.516112 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:19.526544 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d7c38-2321-4af7-9c4c-ebd1983f04ce None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:19.576255 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa44846b-c2a9-44a6-8068-acb040d770de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:19.577130 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44846b-c2a9-44a6-8068-acb040d770de None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:19.577417 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44846b-c2a9-44a6-8068-acb040d770de None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:19.577692 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44846b-c2a9-44a6-8068-acb040d770de None admin] PATH_INFO: `/v3/users/nova` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:19.585829 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:19.588705 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa44846b-c2a9-44a6-8068-acb040d770de None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:19.590380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa44846b-c2a9-44a6-8068-acb040d770de None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:19.612531 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-fa44846b-c2a9-44a6-8068-acb040d770de None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 06 16:55:19.612531 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:19.612531 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:19.612531 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:19.612531 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:19.612531 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:19.612531 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:19.612531 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:19.612531 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:19.612531 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:19.612531 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:19.612531 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:19.612531 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 16:55:19.612531 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 16:55:19.612531 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 16:55:19.612531 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 16:55:19.613621 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:19.613621 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:19.613621 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 16:55:19.613621 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:19.613621 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 16:55:19.613621 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:19.613621 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:19.613621 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:19.613621 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:19.613621 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:19.613621 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:19.613621 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:19.613621 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:19.613621 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:19.613621 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:19.613621 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:19.613621 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:19.613621 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:19.614612 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:19.614612 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:19.614612 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 16:55:19.614612 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 16:55:19.614612 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 16:55:19.614612 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 16:55:19.614612 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 16:55:19.614612 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 16:55:19.614612 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 06 16:55:19.614612 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:19.614612 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 419/812] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Jul 6 16:55:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:19.615190 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa44846b-c2a9-44a6-8068-acb040d770de None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:19.670755 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea44a2e2-07b3-46fb-a47d-8901cbb5347c None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:19.721856 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c9f13b-fbea-487f-975d-812c83df7129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:19.722671 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c9f13b-fbea-487f-975d-812c83df7129 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:19.722957 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c9f13b-fbea-487f-975d-812c83df7129 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:19.723234 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c9f13b-fbea-487f-975d-812c83df7129 None admin] PATH_INFO: `/v3/services/network` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:19.723919 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c9f13b-fbea-487f-975d-812c83df7129 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:19.725290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c9f13b-fbea-487f-975d-812c83df7129 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:19.743270 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-f2c9f13b-fbea-487f-975d-812c83df7129 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 06 16:55:19.743270 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:19.743270 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 16:55:19.743270 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 16:55:19.743270 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 16:55:19.743270 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 16:55:19.743270 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 16:55:19.743270 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:19.743270 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 06 16:55:19.743270 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:19.743270 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 16:55:19.743270 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:19.743270 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:19.743270 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:19.743270 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:19.743270 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:19.743270 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:19.744290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:19.744290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:19.744290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:19.744290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:19.744290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:19.744290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:19.744290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 16:55:19.744290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 16:55:19.744290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 16:55:19.744290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 16:55:19.744290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:19.744290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:19.744290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:19.744290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:19.744290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:19.744290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:19.745145 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:19.745145 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:19.745145 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:19.745145 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:19.745145 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:19.745145 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:19.745145 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:19.745145 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:19.745145 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:19.745145 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:19.745145 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 16:55:19.745145 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:19.745145 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 06 16:55:19.745145 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:19.745861 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 394/813] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:55:19 2022] GET /identity/v3/services/network => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:19.746470 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c9f13b-fbea-487f-975d-812c83df7129 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:19.761590 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa44846b-c2a9-44a6-8068-acb040d770de None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:19.802180 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48847a7a-a13b-4a3f-bc16-906920942030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:19.803012 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48847a7a-a13b-4a3f-bc16-906920942030 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:19.803307 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48847a7a-a13b-4a3f-bc16-906920942030 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:19.803586 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48847a7a-a13b-4a3f-bc16-906920942030 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:19.819498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48847a7a-a13b-4a3f-bc16-906920942030 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:19.822300 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48847a7a-a13b-4a3f-bc16-906920942030 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:19.844885 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-48847a7a-a13b-4a3f-bc16-906920942030 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:19.844885 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:19.844885 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:19.844885 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:19.844885 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:19.844885 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:19.844885 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:19.844885 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:19.844885 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:19.844885 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:19.844885 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:19.844885 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:19.844885 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:55:19.844885 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:55:19.844885 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:55:19.845809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:55:19.845809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:19.845809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:19.845809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:19.845809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:19.845809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:19.845809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:19.845809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:19.845809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:19.845809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:19.845809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:19.845809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:19.845809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:19.845809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:19.845809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:19.845809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:19.845809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:19.846665 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:55:19.846665 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:55:19.846665 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:19.846665 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:19.851916 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 420/814] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:19.851916 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48847a7a-a13b-4a3f-bc16-906920942030 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:19.854058 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:19.876689 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c9f13b-fbea-487f-975d-812c83df7129 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:19.888034 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:19.925476 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820f62d4-9134-4753-b3b2-c7aa6fff1559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:19.927321 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820f62d4-9134-4753-b3b2-c7aa6fff1559 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:19.927844 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820f62d4-9134-4753-b3b2-c7aa6fff1559 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:19.928346 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820f62d4-9134-4753-b3b2-c7aa6fff1559 None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:19.929423 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820f62d4-9134-4753-b3b2-c7aa6fff1559 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:19.929954 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820f62d4-9134-4753-b3b2-c7aa6fff1559 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:19.932142 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820f62d4-9134-4753-b3b2-c7aa6fff1559 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:19.956652 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 395/815] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Jul 6 16:55:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:19.957401 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48847a7a-a13b-4a3f-bc16-906920942030 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:19.958066 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820f62d4-9134-4753-b3b2-c7aa6fff1559 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:20.002980 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81436f2e-bc2e-40d3-996e-e2db506ce547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:20.003811 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81436f2e-bc2e-40d3-996e-e2db506ce547 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:20.004104 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81436f2e-bc2e-40d3-996e-e2db506ce547 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:20.004388 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81436f2e-bc2e-40d3-996e-e2db506ce547 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:20.005367 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81436f2e-bc2e-40d3-996e-e2db506ce547 None admin] RBAC: Adding query filter params (name=network) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:20.005684 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81436f2e-bc2e-40d3-996e-e2db506ce547 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:20.008564 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81436f2e-bc2e-40d3-996e-e2db506ce547 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:20.014418 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 421/816] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Jul 6 16:55:19 2022] GET /identity/v3/services?name=network => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:20.015795 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81436f2e-bc2e-40d3-996e-e2db506ce547 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:20.085921 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820f62d4-9134-4753-b3b2-c7aa6fff1559 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:20.113766 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:20.122948 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891a4ed-377e-4f64-a1ef-755d32731e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:20.127870 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891a4ed-377e-4f64-a1ef-755d32731e80 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:20.127870 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891a4ed-377e-4f64-a1ef-755d32731e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:20.127870 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6891a4ed-377e-4f64-a1ef-755d32731e80 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:20.127870 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6891a4ed-377e-4f64-a1ef-755d32731e80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:20.127870 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6891a4ed-377e-4f64-a1ef-755d32731e80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:20.127870 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6891a4ed-377e-4f64-a1ef-755d32731e80 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:20.141373 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 396/817] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:55:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:20.142486 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891a4ed-377e-4f64-a1ef-755d32731e80 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:20.161523 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81436f2e-bc2e-40d3-996e-e2db506ce547 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:20.171179 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:20.208864 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c609de12-c6c8-4834-83b6-3d2d03c88f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:20.210019 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c609de12-c6c8-4834-83b6-3d2d03c88f82 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:20.210999 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c609de12-c6c8-4834-83b6-3d2d03c88f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:20.211809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c609de12-c6c8-4834-83b6-3d2d03c88f82 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:20.252887 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6891a4ed-377e-4f64-a1ef-755d32731e80 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:20.292639 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4896d7c4-c146-4c9f-8747-eb502615d3ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:20.293468 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896d7c4-c146-4c9f-8747-eb502615d3ff None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:20.293851 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896d7c4-c146-4c9f-8747-eb502615d3ff None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:20.294470 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896d7c4-c146-4c9f-8747-eb502615d3ff None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:20.295075 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4896d7c4-c146-4c9f-8747-eb502615d3ff None admin] RBAC: Adding query filter params (type=network) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:20.295503 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4896d7c4-c146-4c9f-8747-eb502615d3ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:20.297087 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4896d7c4-c146-4c9f-8747-eb502615d3ff None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:20.302649 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 397/818] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Jul 6 16:55:20 2022] GET /identity/v3/services?type=network => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:20.303901 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4896d7c4-c146-4c9f-8747-eb502615d3ff None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:20.306099 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c609de12-c6c8-4834-83b6-3d2d03c88f82 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:20.308922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c609de12-c6c8-4834-83b6-3d2d03c88f82 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:20.386817 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:20.404447 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:20.417533 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4896d7c4-c146-4c9f-8747-eb502615d3ff None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:20.441146 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 422/819] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:55:20 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 16:55:20.441146 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c609de12-c6c8-4834-83b6-3d2d03c88f82 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:20.461775 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0ce4af-6529-47e3-8d85-5511d874e237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:20.463331 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0ce4af-6529-47e3-8d85-5511d874e237 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:20.463898 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0ce4af-6529-47e3-8d85-5511d874e237 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:20.464407 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0ce4af-6529-47e3-8d85-5511d874e237 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:20.475315 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0ce4af-6529-47e3-8d85-5511d874e237 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:20.477807 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0ce4af-6529-47e3-8d85-5511d874e237 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:20.502499 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-df0ce4af-6529-47e3-8d85-5511d874e237 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 06 16:55:20.502499 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:20.502499 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:20.502499 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:20.502499 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:20.502499 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:20.502499 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:20.502499 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:20.502499 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:20.502499 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:20.502499 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:20.502499 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:20.502499 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 16:55:20.502499 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 16:55:20.502499 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 16:55:20.502499 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 16:55:20.503487 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:20.503487 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:20.503487 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 16:55:20.503487 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:20.503487 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 16:55:20.503487 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:20.503487 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:20.503487 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:20.503487 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:20.503487 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:20.503487 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:20.503487 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:20.503487 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:20.503487 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:20.503487 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:20.503487 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:20.503487 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:20.503487 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:20.504499 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:20.504499 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:20.504499 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 16:55:20.504499 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 16:55:20.504499 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 16:55:20.504499 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 16:55:20.504499 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 16:55:20.504499 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 16:55:20.504499 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 06 16:55:20.504499 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:20.505654 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 398/820] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Jul 6 16:55:20 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:20.506406 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0ce4af-6529-47e3-8d85-5511d874e237 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:20.553330 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c609de12-c6c8-4834-83b6-3d2d03c88f82 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:20.598177 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b122b358-5d85-4ef7-af3b-e33666afb73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:20.599222 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122b358-5d85-4ef7-af3b-e33666afb73c None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:20.599767 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122b358-5d85-4ef7-af3b-e33666afb73c None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:20.600255 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b122b358-5d85-4ef7-af3b-e33666afb73c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:20.601754 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b122b358-5d85-4ef7-af3b-e33666afb73c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:20.603929 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b122b358-5d85-4ef7-af3b-e33666afb73c None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:20.628125 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0ce4af-6529-47e3-8d85-5511d874e237 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:20.670712 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c9023f-e562-4989-8adb-9d0f31bbf5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:20.671576 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9023f-e562-4989-8adb-9d0f31bbf5e4 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:20.671989 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9023f-e562-4989-8adb-9d0f31bbf5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:20.672267 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c9023f-e562-4989-8adb-9d0f31bbf5e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:20.684079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c9023f-e562-4989-8adb-9d0f31bbf5e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:20.685670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c9023f-e562-4989-8adb-9d0f31bbf5e4 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:20.690193 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 423/821] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 6 16:55:20 2022] POST /identity/v3/endpoints => generated 327 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 16:55:20.691520 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b122b358-5d85-4ef7-af3b-e33666afb73c None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:20.697100 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:20.707755 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-37c9023f-e562-4989-8adb-9d0f31bbf5e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:20.707755 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:20.707755 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:20.707755 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:20.707755 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:20.707755 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:20.707755 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:20.707755 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:20.707755 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:20.707755 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:20.707755 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:20.707755 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:20.707755 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:55:20.707755 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:55:20.707755 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:55:20.708771 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:55:20.708771 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:20.708771 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:20.708771 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:20.708771 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:20.708771 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:20.708771 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:20.708771 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:20.708771 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:20.708771 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:20.708771 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:20.708771 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:20.708771 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:20.708771 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:20.708771 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:20.708771 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:20.708771 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:20.709660 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:55:20.709660 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:55:20.709660 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:20.709660 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:20.713851 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 399/822] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:20.715271 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c9023f-e562-4989-8adb-9d0f31bbf5e4 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:20.731652 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:20.844593 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b122b358-5d85-4ef7-af3b-e33666afb73c None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:20.884956 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a226335-20e6-4e58-a3a6-fbb2848641e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:20.885645 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a226335-20e6-4e58-a3a6-fbb2848641e0 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:20.885934 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a226335-20e6-4e58-a3a6-fbb2848641e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:20.886214 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a226335-20e6-4e58-a3a6-fbb2848641e0 None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:20.887060 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a226335-20e6-4e58-a3a6-fbb2848641e0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:20.887371 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a226335-20e6-4e58-a3a6-fbb2848641e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:20.888853 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a226335-20e6-4e58-a3a6-fbb2848641e0 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:20.910772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 424/823] 10.222.0.40 () {56 vars in 1149 bytes} [Wed Jul 6 16:55:20 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:20.916100 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a226335-20e6-4e58-a3a6-fbb2848641e0 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:20.950470 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c9023f-e562-4989-8adb-9d0f31bbf5e4 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:20.998692 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a76aea-753c-4bc9-8538-3c30aeea66fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:20.999365 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a76aea-753c-4bc9-8538-3c30aeea66fb None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:20.999651 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a76aea-753c-4bc9-8538-3c30aeea66fb None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:20.999973 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a76aea-753c-4bc9-8538-3c30aeea66fb None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:21.000705 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a76aea-753c-4bc9-8538-3c30aeea66fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:21.001010 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a76aea-753c-4bc9-8538-3c30aeea66fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:21.002381 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a76aea-753c-4bc9-8538-3c30aeea66fb None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:21.015535 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:21.021816 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 400/824] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:55:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:21.023152 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a76aea-753c-4bc9-8538-3c30aeea66fb None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:21.044930 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a226335-20e6-4e58-a3a6-fbb2848641e0 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:21.086537 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a1b0ef-a4f6-42e6-bd42-953ebf010e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:21.087343 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1b0ef-a4f6-42e6-bd42-953ebf010e0c None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:21.087626 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1b0ef-a4f6-42e6-bd42-953ebf010e0c None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:21.087924 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a1b0ef-a4f6-42e6-bd42-953ebf010e0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:21.102307 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a1b0ef-a4f6-42e6-bd42-953ebf010e0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:21.103784 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a1b0ef-a4f6-42e6-bd42-953ebf010e0c None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:21.103964 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:21.121896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-b6a1b0ef-a4f6-42e6-bd42-953ebf010e0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:21.121896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:21.121896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:21.121896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:21.121896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:21.121896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:21.121896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:21.121896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:21.121896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:21.121896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:21.121896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:21.121896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:21.121896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:55:21.121896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:55:21.121896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:55:21.123030 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:55:21.123030 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:21.123030 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:21.123030 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:21.123030 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:21.123030 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:21.123030 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:21.123030 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:21.123030 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:21.123030 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:21.123030 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:21.123030 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:21.123030 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:21.123030 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:21.123030 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:21.123030 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:21.123030 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:21.123951 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:55:21.123951 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:55:21.123951 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:21.123951 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:21.123951 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 425/825] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:55:21.124786 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ea939-a2ce-4e53-b596-579d33856b67 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:21.125078 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ea939-a2ce-4e53-b596-579d33856b67 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:21.125360 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90ea939-a2ce-4e53-b596-579d33856b67 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:21.126751 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 426/826] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:21.128530 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ea939-a2ce-4e53-b596-579d33856b67 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:21.152556 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a76aea-753c-4bc9-8538-3c30aeea66fb None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:21.185455 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ec299b-c3df-43f3-a934-654eaf2ab802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:21.186250 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec299b-c3df-43f3-a934-654eaf2ab802 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:21.186535 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec299b-c3df-43f3-a934-654eaf2ab802 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:21.186808 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec299b-c3df-43f3-a934-654eaf2ab802 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:21.193512 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ec299b-c3df-43f3-a934-654eaf2ab802 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:21.195481 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ec299b-c3df-43f3-a934-654eaf2ab802 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:21.209944 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-21ec299b-c3df-43f3-a934-654eaf2ab802 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:21.209944 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:21.209944 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:21.209944 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:21.209944 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:21.209944 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:21.209944 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:21.209944 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:21.209944 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:21.209944 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:21.209944 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:21.209944 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:21.209944 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 16:55:21.209944 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 16:55:21.209944 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 16:55:21.210882 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 16:55:21.210882 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:21.210882 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:21.210882 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:21.210882 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:21.210882 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:21.210882 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:21.210882 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:21.210882 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:21.210882 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:21.210882 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:21.210882 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:21.210882 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:21.210882 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:21.210882 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:21.210882 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:21.210882 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:21.211800 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 16:55:21.211800 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 16:55:21.211800 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 16:55:21.211800 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 16:55:21.211800 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 16:55:21.211800 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 16:55:21.211800 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:21.211800 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:21.211800 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 401/827] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:55:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:21.212583 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f5b15-dbc0-4b69-b7d1-92816634bc47 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:21.212687 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f5b15-dbc0-4b69-b7d1-92816634bc47 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:21.212920 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363f5b15-dbc0-4b69-b7d1-92816634bc47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:21.229490 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-363f5b15-dbc0-4b69-b7d1-92816634bc47 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:21.230667 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-363f5b15-dbc0-4b69-b7d1-92816634bc47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:21.244893 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90ea939-a2ce-4e53-b596-579d33856b67 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:21.275299 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:21.283873 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3cd80c-0530-406c-b90d-72cb377d6fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:21.284541 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3cd80c-0530-406c-b90d-72cb377d6fee None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:21.284825 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3cd80c-0530-406c-b90d-72cb377d6fee None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:21.285097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3cd80c-0530-406c-b90d-72cb377d6fee None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:21.285942 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3cd80c-0530-406c-b90d-72cb377d6fee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:21.286246 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3cd80c-0530-406c-b90d-72cb377d6fee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:21.287603 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3cd80c-0530-406c-b90d-72cb377d6fee None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:21.298820 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 402/828] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:21 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:55:21.304879 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 427/829] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:55:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:21.306334 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f5b15-dbc0-4b69-b7d1-92816634bc47 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:21.306442 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3cd80c-0530-406c-b90d-72cb377d6fee None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:21.369831 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:21.418286 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3cd80c-0530-406c-b90d-72cb377d6fee None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:21.421924 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363f5b15-dbc0-4b69-b7d1-92816634bc47 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:21.452690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d652d9-3846-4d9f-aeb6-4d2171b9e9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:21.453479 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d652d9-3846-4d9f-aeb6-4d2171b9e9c9 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:21.453761 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d652d9-3846-4d9f-aeb6-4d2171b9e9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:21.454032 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d652d9-3846-4d9f-aeb6-4d2171b9e9c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:21.461268 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d652d9-3846-4d9f-aeb6-4d2171b9e9c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:21.461675 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca6cf6c-65d0-42ee-899b-c617e12ec64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:21.462555 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6cf6c-65d0-42ee-899b-c617e12ec64b None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:21.462665 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d652d9-3846-4d9f-aeb6-4d2171b9e9c9 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:21.463139 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6cf6c-65d0-42ee-899b-c617e12ec64b None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:21.463610 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6cf6c-65d0-42ee-899b-c617e12ec64b None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:21.464788 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca6cf6c-65d0-42ee-899b-c617e12ec64b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:21.465281 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca6cf6c-65d0-42ee-899b-c617e12ec64b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:21.466825 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca6cf6c-65d0-42ee-899b-c617e12ec64b None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:21.478642 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-30d652d9-3846-4d9f-aeb6-4d2171b9e9c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:21.478642 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:21.478642 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:21.478642 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:21.478642 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:21.478642 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:21.478642 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:21.478642 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:21.478642 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:21.478642 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:21.478642 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:21.478642 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:21.478642 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 16:55:21.478642 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 16:55:21.478642 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 16:55:21.488825 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 16:55:21.488825 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:21.488825 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:21.488825 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:21.488825 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:21.488825 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:21.488825 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:21.488825 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:21.488825 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:21.488825 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:21.488825 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:21.488825 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:21.488825 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:21.488825 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:21.488825 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:21.488825 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:21.488825 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:21.489694 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 16:55:21.489694 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 16:55:21.489694 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 16:55:21.489694 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 16:55:21.489694 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 16:55:21.489694 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 16:55:21.489694 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:21.489694 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:21.490187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 403/830] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 6 16:55:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:21.490187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca6cf6c-65d0-42ee-899b-c617e12ec64b None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:21.490321 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 428/831] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:55:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:21.490321 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d652d9-3846-4d9f-aeb6-4d2171b9e9c9 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:21.540718 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:21.594544 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d652d9-3846-4d9f-aeb6-4d2171b9e9c9 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:21.595893 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca6cf6c-65d0-42ee-899b-c617e12ec64b None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:21.607891 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:21.633899 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e28a6c5-0a95-4b50-a632-21a84ded514c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:21.634444 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51cc916-c75e-49cb-b05d-f3c48dc80af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:21.634704 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28a6c5-0a95-4b50-a632-21a84ded514c None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:21.634987 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28a6c5-0a95-4b50-a632-21a84ded514c None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:21.635084 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51cc916-c75e-49cb-b05d-f3c48dc80af1 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:21.635256 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e28a6c5-0a95-4b50-a632-21a84ded514c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:21.635407 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51cc916-c75e-49cb-b05d-f3c48dc80af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:21.635638 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51cc916-c75e-49cb-b05d-f3c48dc80af1 None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:21.635957 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e28a6c5-0a95-4b50-a632-21a84ded514c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:21.636451 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51cc916-c75e-49cb-b05d-f3c48dc80af1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:21.636755 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51cc916-c75e-49cb-b05d-f3c48dc80af1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:21.637445 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e28a6c5-0a95-4b50-a632-21a84ded514c None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:21.638287 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51cc916-c75e-49cb-b05d-f3c48dc80af1 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:21.650417 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-6e28a6c5-0a95-4b50-a632-21a84ded514c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 06 16:55:21.650417 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:21.650417 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 16:55:21.650417 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 16:55:21.650417 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 16:55:21.650417 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 16:55:21.650417 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 16:55:21.650417 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:21.650417 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 06 16:55:21.650417 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:21.650417 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 16:55:21.650417 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:21.650417 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:21.650417 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:21.650417 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:21.650417 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:21.650417 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:21.651594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:21.651594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:21.651594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:21.651594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:21.651594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:21.651594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:21.651594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 16:55:21.651594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 16:55:21.651594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 16:55:21.651594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 16:55:21.651594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:21.651594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:21.651594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:21.651594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:21.651594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:21.651594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:21.652644 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:21.652644 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:21.652644 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:21.652644 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:21.652644 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:21.652644 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:21.652644 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:21.652644 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:21.652644 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:21.652644 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:21.652644 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 16:55:21.652644 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:21.652644 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 06 16:55:21.652644 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:21.655123 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 404/832] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 6 16:55:21 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:21.660854 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e28a6c5-0a95-4b50-a632-21a84ded514c None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:21.660979 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 429/833] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 6 16:55:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:21.660979 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51cc916-c75e-49cb-b05d-f3c48dc80af1 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:21.761561 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e28a6c5-0a95-4b50-a632-21a84ded514c None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:21.761762 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51cc916-c75e-49cb-b05d-f3c48dc80af1 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:21.771438 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:21.795861 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc1ce0f-21a6-4cac-ac4c-6eafab6fbb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:21.796055 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735efe43-6918-4fbf-9d48-f246afdba88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:21.796678 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc1ce0f-21a6-4cac-ac4c-6eafab6fbb89 None admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:21.796786 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735efe43-6918-4fbf-9d48-f246afdba88a None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:21.796959 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc1ce0f-21a6-4cac-ac4c-6eafab6fbb89 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:21.797062 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735efe43-6918-4fbf-9d48-f246afdba88a None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:21.797232 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc1ce0f-21a6-4cac-ac4c-6eafab6fbb89 None admin] PATH_INFO: `/v3/projects/7938b0346d4240ab89774327d09a89ae/users/1905e6f8c3e24e3cbfefce2827baa370/roles/750bca9c32ac4aa49adcbee30463f1fa` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:21.797324 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735efe43-6918-4fbf-9d48-f246afdba88a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:21.798180 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735efe43-6918-4fbf-9d48-f246afdba88a None admin] RBAC: Adding query filter params (role.id=b41f8a7a58e64f749f2099ca45cc04a7, scope.project.id=7938b0346d4240ab89774327d09a89ae, user.id=af26ef96dc9d47dc91549b736939268e) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:21.798493 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735efe43-6918-4fbf-9d48-f246afdba88a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:21.799946 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735efe43-6918-4fbf-9d48-f246afdba88a None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:21.805793 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 405/834] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:55:21 2022] GET /identity/v3/role_assignments?user.id=af26ef96dc9d47dc91549b736939268e&scope.project.id=7938b0346d4240ab89774327d09a89ae&role.id=b41f8a7a58e64f749f2099ca45cc04a7 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:21.806624 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735efe43-6918-4fbf-9d48-f246afdba88a None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:21.819100 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc1ce0f-21a6-4cac-ac4c-6eafab6fbb89 None admin] RBAC: Authorizing `identity:create_grant(project_id=7938b0346d4240ab89774327d09a89ae, user_id=1905e6f8c3e24e3cbfefce2827baa370, role_id=750bca9c32ac4aa49adcbee30463f1fa)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:21.820704 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc1ce0f-21a6-4cac-ac4c-6eafab6fbb89 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:21.850726 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:21.907755 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 430/835] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 6 16:55:21 2022] PUT /identity/v3/projects/7938b0346d4240ab89774327d09a89ae/users/1905e6f8c3e24e3cbfefce2827baa370/roles/750bca9c32ac4aa49adcbee30463f1fa => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 16:55:21.907755 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d968e73-2643-4e26-a688-9f6d3fcf4270 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:21.907755 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d968e73-2643-4e26-a688-9f6d3fcf4270 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:21.907755 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d968e73-2643-4e26-a688-9f6d3fcf4270 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:21.907755 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 431/836] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:21.907755 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b53cad-f7ee-40a2-8647-6350e4fbdf7f None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:21.908289 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b53cad-f7ee-40a2-8647-6350e4fbdf7f None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:21.908289 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b53cad-f7ee-40a2-8647-6350e4fbdf7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:21.927153 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-36b53cad-f7ee-40a2-8647-6350e4fbdf7f None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:21.928438 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-36b53cad-f7ee-40a2-8647-6350e4fbdf7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:21.931111 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735efe43-6918-4fbf-9d48-f246afdba88a None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:21.987514 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e7589a-6bb8-4b96-b92d-b1adad337594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:21.987514 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e7589a-6bb8-4b96-b92d-b1adad337594 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:21.987514 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e7589a-6bb8-4b96-b92d-b1adad337594 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:21.987514 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e7589a-6bb8-4b96-b92d-b1adad337594 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:21.987935 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e7589a-6bb8-4b96-b92d-b1adad337594 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:21.987935 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e7589a-6bb8-4b96-b92d-b1adad337594 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:21.988579 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e7589a-6bb8-4b96-b92d-b1adad337594 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:21.997588 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 406/837] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 6 16:55:21 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:22.003222 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e7589a-6bb8-4b96-b92d-b1adad337594 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:22.033071 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 432/838] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:21 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 16:55:22.056329 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b53cad-f7ee-40a2-8647-6350e4fbdf7f None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:22.133764 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e7589a-6bb8-4b96-b92d-b1adad337594 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:22.135859 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:22.171831 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:22.177742 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ffc1d-78da-4f65-b3c7-cb4d9acdb06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:22.178843 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ffc1d-78da-4f65-b3c7-cb4d9acdb06d None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:22.179364 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ffc1d-78da-4f65-b3c7-cb4d9acdb06d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:22.180162 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4ffc1d-78da-4f65-b3c7-cb4d9acdb06d None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:22.181307 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4ffc1d-78da-4f65-b3c7-cb4d9acdb06d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:22.181970 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4ffc1d-78da-4f65-b3c7-cb4d9acdb06d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:22.184143 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4ffc1d-78da-4f65-b3c7-cb4d9acdb06d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:22.185732 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b53cad-f7ee-40a2-8647-6350e4fbdf7f None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:22.190453 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 407/839] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 6 16:55:22 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:22.226858 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad688be7-911f-4b8c-adf5-2872803d83ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:22.227920 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad688be7-911f-4b8c-adf5-2872803d83ae None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:22.228280 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad688be7-911f-4b8c-adf5-2872803d83ae None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:22.228647 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad688be7-911f-4b8c-adf5-2872803d83ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:22.245799 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad688be7-911f-4b8c-adf5-2872803d83ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:55:22.254187 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-ad688be7-911f-4b8c-adf5-2872803d83ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:55:22.254187 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:22.254187 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:22.254187 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:22.254187 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:22.254187 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:22.254187 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:22.254187 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:22.254187 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:22.254187 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:22.254187 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:22.254187 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:22.254187 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:55:22.254187 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:55:22.254187 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:55:22.254187 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:55:22.255197 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:55:22.255197 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:55:22.255197 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:55:22.255197 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:55:22.255197 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:22.255197 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:22.255197 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:22.255197 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:22.255197 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:22.255197 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:22.255197 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:22.255197 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:22.255197 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:22.255197 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:22.255197 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:22.255197 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:22.256045 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:22.256045 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:22.256045 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:22.256045 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:22.256045 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:55:22.256045 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:55:22.256045 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:55:22.256045 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:55:22.256045 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:55:22.256045 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:55:22.256045 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:55:22.256045 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:22.259276 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 433/840] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 6 16:55:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:22.265754 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ffc1d-78da-4f65-b3c7-cb4d9acdb06d None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:22.379214 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4ffc1d-78da-4f65-b3c7-cb4d9acdb06d None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:22.415761 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8bdbc-a08b-47ba-ac99-4962e24154d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:22.416706 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8bdbc-a08b-47ba-ac99-4962e24154d8 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:22.417136 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8bdbc-a08b-47ba-ac99-4962e24154d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:22.417566 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8bdbc-a08b-47ba-ac99-4962e24154d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:22.418462 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa8bdbc-a08b-47ba-ac99-4962e24154d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:22.418910 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa8bdbc-a08b-47ba-ac99-4962e24154d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:22.420439 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa8bdbc-a08b-47ba-ac99-4962e24154d8 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:22.430060 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 408/841] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:55:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:22.435769 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad688be7-911f-4b8c-adf5-2872803d83ae None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:22.542350 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad688be7-911f-4b8c-adf5-2872803d83ae None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:22.579057 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0af87e-0575-41a1-89e2-2234c0a7a09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:22.580210 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0af87e-0575-41a1-89e2-2234c0a7a09e None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:22.580639 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0af87e-0575-41a1-89e2-2234c0a7a09e None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:22.581054 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0af87e-0575-41a1-89e2-2234c0a7a09e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:22.592452 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0af87e-0575-41a1-89e2-2234c0a7a09e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:22.594112 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0af87e-0575-41a1-89e2-2234c0a7a09e None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:22.603486 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:22.613794 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-2f0af87e-0575-41a1-89e2-2234c0a7a09e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:22.613794 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:22.613794 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:22.613794 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:22.613794 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:22.613794 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:22.613794 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:22.613794 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:22.613794 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:22.613794 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:22.613794 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:22.613794 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:22.613794 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:55:22.613794 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:55:22.613794 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:55:22.620226 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:55:22.620226 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:22.620226 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:22.620226 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:22.620226 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:22.620226 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:22.620226 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:22.620226 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:22.620226 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:22.620226 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:22.620226 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:22.620226 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:22.620226 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:22.620226 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:22.620226 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:22.620226 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:22.620226 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:22.621404 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:55:22.621404 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:55:22.621404 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:22.621404 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:22.621404 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 434/842] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:22.624187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8bdbc-a08b-47ba-ac99-4962e24154d8 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:22.653773 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:22.729038 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa8bdbc-a08b-47ba-ac99-4962e24154d8 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:22.769650 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f56c6a-dc81-4463-9b2f-d269a3a67e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:22.770594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f56c6a-dc81-4463-9b2f-d269a3a67e41 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:22.771020 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f56c6a-dc81-4463-9b2f-d269a3a67e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:22.771743 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f56c6a-dc81-4463-9b2f-d269a3a67e41 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:22.772604 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f56c6a-dc81-4463-9b2f-d269a3a67e41 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:22.773090 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f56c6a-dc81-4463-9b2f-d269a3a67e41 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:22.774599 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f56c6a-dc81-4463-9b2f-d269a3a67e41 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:22.786358 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 409/843] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:55:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:22.792629 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0af87e-0575-41a1-89e2-2234c0a7a09e None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:22.894192 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0af87e-0575-41a1-89e2-2234c0a7a09e None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:22.929660 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ebc9c6-19f9-4355-9202-b8521d9b5be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:22.930701 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebc9c6-19f9-4355-9202-b8521d9b5be2 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:22.931131 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebc9c6-19f9-4355-9202-b8521d9b5be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:22.931575 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ebc9c6-19f9-4355-9202-b8521d9b5be2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:22.939478 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ebc9c6-19f9-4355-9202-b8521d9b5be2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:22.941092 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ebc9c6-19f9-4355-9202-b8521d9b5be2 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:22.955453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-e0ebc9c6-19f9-4355-9202-b8521d9b5be2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 06 16:55:22.955453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:22.955453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:22.955453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:22.955453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:22.955453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:22.955453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:22.955453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:22.955453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:22.955453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:22.955453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:22.955453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:22.955453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 16:55:22.955453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 16:55:22.955453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 16:55:22.955453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 16:55:22.956620 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:22.956620 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:22.956620 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 16:55:22.956620 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:22.956620 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 16:55:22.956620 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:22.956620 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:22.956620 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:22.956620 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:22.956620 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:22.956620 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:22.956620 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:22.956620 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:22.956620 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:22.956620 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:22.956620 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:22.956620 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:22.956620 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:22.957774 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:22.957774 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:22.957774 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 16:55:22.957774 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 16:55:22.957774 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 16:55:22.957774 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 16:55:22.957774 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 16:55:22.957774 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 16:55:22.957774 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 06 16:55:22.957774 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:22.959929 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 435/844] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 6 16:55:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:22.965461 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f56c6a-dc81-4463-9b2f-d269a3a67e41 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:23.085330 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:23.098319 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f56c6a-dc81-4463-9b2f-d269a3a67e41 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:23.133846 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba27095-bc46-4bbb-a5e1-e2326717016a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:23.134527 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba27095-bc46-4bbb-a5e1-e2326717016a None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:23.134816 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba27095-bc46-4bbb-a5e1-e2326717016a None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:23.135093 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba27095-bc46-4bbb-a5e1-e2326717016a None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:23.135964 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba27095-bc46-4bbb-a5e1-e2326717016a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:23.136279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba27095-bc46-4bbb-a5e1-e2326717016a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:23.137713 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba27095-bc46-4bbb-a5e1-e2326717016a None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:23.156005 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 410/845] 10.222.0.40 () {56 vars in 1135 bytes} [Wed Jul 6 16:55:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:23.161197 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ebc9c6-19f9-4355-9202-b8521d9b5be2 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:23.227148 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:23.268363 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ebc9c6-19f9-4355-9202-b8521d9b5be2 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:23.302072 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f14e75-4800-4255-8ddc-24d52067eafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:23.302732 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f14e75-4800-4255-8ddc-24d52067eafe None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:23.303014 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f14e75-4800-4255-8ddc-24d52067eafe None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:23.303293 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f14e75-4800-4255-8ddc-24d52067eafe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:23.371855 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f14e75-4800-4255-8ddc-24d52067eafe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:23.373698 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f14e75-4800-4255-8ddc-24d52067eafe None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:23.438524 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:23.477945 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 436/846] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:55:23 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 16:55:23.483221 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba27095-bc46-4bbb-a5e1-e2326717016a None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:23.591710 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba27095-bc46-4bbb-a5e1-e2326717016a None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:23.630233 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42af64e-de5e-4c0a-8d73-9407dfd97f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:23.631363 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42af64e-de5e-4c0a-8d73-9407dfd97f24 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:23.631837 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42af64e-de5e-4c0a-8d73-9407dfd97f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:23.632280 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42af64e-de5e-4c0a-8d73-9407dfd97f24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:23.644321 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42af64e-de5e-4c0a-8d73-9407dfd97f24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:23.645770 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42af64e-de5e-4c0a-8d73-9407dfd97f24 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:23.656093 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:23.665187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-d42af64e-de5e-4c0a-8d73-9407dfd97f24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:23.665187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:23.665187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:23.665187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:23.665187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:23.665187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:23.665187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:23.665187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:23.665187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:23.665187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:23.665187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:23.665187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:23.665187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:55:23.665187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:55:23.665187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:55:23.673097 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:55:23.673097 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:23.673097 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:23.673097 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:23.673097 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:23.673097 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:23.673097 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:23.673097 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:23.673097 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:23.673097 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:23.673097 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:23.673097 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:23.673097 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:23.673097 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:23.673097 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:23.673097 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:23.673097 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:23.674014 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:55:23.674014 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:55:23.674014 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:23.674014 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:23.674014 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 411/847] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:23.674299 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f14e75-4800-4255-8ddc-24d52067eafe None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:23.778217 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f14e75-4800-4255-8ddc-24d52067eafe None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:23.816177 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeccbe9-7288-46e6-b958-edf72073844f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:23.816891 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeccbe9-7288-46e6-b958-edf72073844f None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:23.817175 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeccbe9-7288-46e6-b958-edf72073844f None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:23.817450 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeccbe9-7288-46e6-b958-edf72073844f None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:23.818173 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeccbe9-7288-46e6-b958-edf72073844f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:23.818473 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeccbe9-7288-46e6-b958-edf72073844f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:23.819896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eeccbe9-7288-46e6-b958-edf72073844f None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:23.832333 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 437/848] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:55:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:23.838391 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42af64e-de5e-4c0a-8d73-9407dfd97f24 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:23.947423 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42af64e-de5e-4c0a-8d73-9407dfd97f24 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:23.988206 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508dbc28-8c38-425d-8d90-eee060344450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:23.989014 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508dbc28-8c38-425d-8d90-eee060344450 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:23.989301 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508dbc28-8c38-425d-8d90-eee060344450 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:23.989581 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508dbc28-8c38-425d-8d90-eee060344450 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:23.996751 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508dbc28-8c38-425d-8d90-eee060344450 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:23.998207 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508dbc28-8c38-425d-8d90-eee060344450 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:24.014039 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-508dbc28-8c38-425d-8d90-eee060344450 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:24.014039 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:24.014039 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:24.014039 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:24.014039 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:24.014039 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:24.014039 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:24.014039 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:24.014039 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:24.014039 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:24.014039 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:24.014039 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:24.014039 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 16:55:24.014039 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 16:55:24.014039 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 16:55:24.014982 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 16:55:24.014982 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:24.014982 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:24.014982 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:24.014982 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:24.014982 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:24.014982 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:24.014982 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:24.014982 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:24.014982 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:24.014982 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:24.014982 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:24.014982 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:24.014982 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:24.014982 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:24.014982 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:24.014982 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:24.015935 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 16:55:24.015935 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 16:55:24.015935 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 16:55:24.015935 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 16:55:24.015935 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 16:55:24.015935 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 16:55:24.015935 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:24.015935 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:24.016423 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 412/849] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:55:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:24.023538 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeccbe9-7288-46e6-b958-edf72073844f None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:24.039295 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17fe313-2a2d-4f19-9245-dbbfae567403 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:24.039587 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17fe313-2a2d-4f19-9245-dbbfae567403 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:24.039902 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17fe313-2a2d-4f19-9245-dbbfae567403 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:24.041692 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 413/850] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 16:55:24.046463 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f7b48-f028-44e6-af36-80d8cf85e496 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:24.046753 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f7b48-f028-44e6-af36-80d8cf85e496 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:24.047047 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95f7b48-f028-44e6-af36-80d8cf85e496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:24.049375 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:24.071755 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-a95f7b48-f028-44e6-af36-80d8cf85e496 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:24.071755 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-a95f7b48-f028-44e6-af36-80d8cf85e496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:24.128620 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:24.142596 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 414/851] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:24 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 16:55:24.143981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06268be2-b8ef-44dd-9cd3-96a0012cfdef None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:24.144216 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06268be2-b8ef-44dd-9cd3-96a0012cfdef None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:24.144491 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06268be2-b8ef-44dd-9cd3-96a0012cfdef None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:24.150499 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 415/852] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:24.151721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec2d5b-2830-4c98-9692-a5c62152bf26 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:24.152039 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec2d5b-2830-4c98-9692-a5c62152bf26 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:24.152315 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec2d5b-2830-4c98-9692-a5c62152bf26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:24.156989 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eeccbe9-7288-46e6-b958-edf72073844f None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:24.175788 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-71ec2d5b-2830-4c98-9692-a5c62152bf26 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:24.176999 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-71ec2d5b-2830-4c98-9692-a5c62152bf26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:24.194266 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a0b26-1355-47c5-a0ec-453bd1b24857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:24.195462 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a0b26-1355-47c5-a0ec-453bd1b24857 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:24.195768 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a0b26-1355-47c5-a0ec-453bd1b24857 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:24.196040 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a0b26-1355-47c5-a0ec-453bd1b24857 None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:24.196815 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341a0b26-1355-47c5-a0ec-453bd1b24857 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:24.197121 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341a0b26-1355-47c5-a0ec-453bd1b24857 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:24.198584 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341a0b26-1355-47c5-a0ec-453bd1b24857 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:24.215566 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 438/853] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 6 16:55:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:24.216455 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a0b26-1355-47c5-a0ec-453bd1b24857 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:24.252036 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 416/854] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:24 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:55:24.252940 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ec2d5b-2830-4c98-9692-a5c62152bf26 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:24.322333 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:24.341747 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a0b26-1355-47c5-a0ec-453bd1b24857 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:24.372855 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ec2d5b-2830-4c98-9692-a5c62152bf26 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:24.378341 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80ebafc-a9d7-4b64-ae03-1e4450c72807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:24.379162 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ebafc-a9d7-4b64-ae03-1e4450c72807 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:24.379447 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ebafc-a9d7-4b64-ae03-1e4450c72807 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:24.379749 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ebafc-a9d7-4b64-ae03-1e4450c72807 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:24.380414 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80ebafc-a9d7-4b64-ae03-1e4450c72807 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:24.381774 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80ebafc-a9d7-4b64-ae03-1e4450c72807 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:24.390343 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:24.393931 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-d80ebafc-a9d7-4b64-ae03-1e4450c72807 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 06 16:55:24.393931 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:24.393931 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 16:55:24.393931 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 16:55:24.393931 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 16:55:24.393931 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 16:55:24.393931 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 16:55:24.393931 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:24.393931 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 06 16:55:24.393931 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:24.393931 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 16:55:24.393931 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:24.393931 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:24.393931 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:24.393931 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:24.393931 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:24.394848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:24.394848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:24.394848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:24.394848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:24.394848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:24.394848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:24.394848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:24.394848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 16:55:24.394848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 16:55:24.394848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 16:55:24.394848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 16:55:24.394848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:24.394848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:24.394848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:24.394848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:24.394848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:24.395658 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:24.395658 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:24.395658 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:24.395658 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:24.395658 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:24.395658 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:24.395658 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:24.395658 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:24.395658 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:24.395658 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:24.395658 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:24.395658 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 16:55:24.395658 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:24.395658 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 06 16:55:24.395658 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:24.395658 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 439/855] 10.222.0.40 () {56 vars in 1132 bytes} [Wed Jul 6 16:55:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:24.396866 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80ebafc-a9d7-4b64-ae03-1e4450c72807 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:24.407429 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c4137-a02a-42de-b359-96d0523a053e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:24.408126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c4137-a02a-42de-b359-96d0523a053e None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:24.408413 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c4137-a02a-42de-b359-96d0523a053e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:24.410399 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c4137-a02a-42de-b359-96d0523a053e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:24.410399 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c4137-a02a-42de-b359-96d0523a053e None admin] RBAC: Adding query filter params (role.id=b41f8a7a58e64f749f2099ca45cc04a7, scope.project.id=7938b0346d4240ab89774327d09a89ae, user.id=75acacd30e9a4cf2ad58b072b49e3f05) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:24.410399 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c4137-a02a-42de-b359-96d0523a053e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:24.411209 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c4137-a02a-42de-b359-96d0523a053e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:24.417420 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 417/856] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:55:24 2022] GET /identity/v3/role_assignments?user.id=75acacd30e9a4cf2ad58b072b49e3f05&scope.project.id=7938b0346d4240ab89774327d09a89ae&role.id=b41f8a7a58e64f749f2099ca45cc04a7 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:24.418618 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c516c2d-5ba4-4744-ae93-c5332bb59c73 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:24.418907 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c516c2d-5ba4-4744-ae93-c5332bb59c73 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:24.419185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c516c2d-5ba4-4744-ae93-c5332bb59c73 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:24.420434 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 418/857] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:24.421134 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c516c2d-5ba4-4744-ae93-c5332bb59c73 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:24.512446 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80ebafc-a9d7-4b64-ae03-1e4450c72807 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:24.537995 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c516c2d-5ba4-4744-ae93-c5332bb59c73 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:24.550444 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3542fcaa-806f-44ef-8785-33de350ad36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:24.551268 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542fcaa-806f-44ef-8785-33de350ad36c None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:24.551557 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542fcaa-806f-44ef-8785-33de350ad36c None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:24.551874 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3542fcaa-806f-44ef-8785-33de350ad36c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:24.563134 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3542fcaa-806f-44ef-8785-33de350ad36c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:55:24.571024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-3542fcaa-806f-44ef-8785-33de350ad36c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 16:55:24.571024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:24.571024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:24.571024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:24.571024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:24.571024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:24.571024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:24.571024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:24.571024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:24.571024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:24.571024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:24.571024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:24.571024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:55:24.571024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:55:24.571024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:55:24.571024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:55:24.571991 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:55:24.571991 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:55:24.571991 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:55:24.571991 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:55:24.571991 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:24.571991 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:24.571991 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:24.571991 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:24.571991 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:24.571991 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:24.571991 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:24.571991 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:24.571991 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:24.571991 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:24.571991 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:24.571991 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:24.572808 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:24.572808 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:24.572808 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:24.572808 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:24.572808 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:55:24.572808 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:55:24.572808 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:55:24.572808 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:55:24.572808 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:55:24.572808 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:55:24.572808 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 16:55:24.572808 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:24.579642 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 440/858] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 6 16:55:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:24.579642 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24816337-71ca-4a02-9c4d-bb6dce9b546e None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:24.579642 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24816337-71ca-4a02-9c4d-bb6dce9b546e None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:24.579642 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24816337-71ca-4a02-9c4d-bb6dce9b546e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:24.584932 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145e23a-554b-417f-bdd0-06f7ae08feb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:24.585078 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:24.586340 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145e23a-554b-417f-bdd0-06f7ae08feb7 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:24.586693 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145e23a-554b-417f-bdd0-06f7ae08feb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:24.586979 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7145e23a-554b-417f-bdd0-06f7ae08feb7 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:24.587698 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7145e23a-554b-417f-bdd0-06f7ae08feb7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:24.588041 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7145e23a-554b-417f-bdd0-06f7ae08feb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:24.590489 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7145e23a-554b-417f-bdd0-06f7ae08feb7 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:24.596122 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-24816337-71ca-4a02-9c4d-bb6dce9b546e None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:24.596270 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 419/859] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Jul 6 16:55:24 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:24.597039 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145e23a-554b-417f-bdd0-06f7ae08feb7 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:24.600067 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-24816337-71ca-4a02-9c4d-bb6dce9b546e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:24.654441 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:24.672890 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 441/860] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:24 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:55:24.673645 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24816337-71ca-4a02-9c4d-bb6dce9b546e None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:24.722125 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7145e23a-554b-417f-bdd0-06f7ae08feb7 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:24.763959 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4059bf-c8ac-4e0a-b33b-21e6a7adc278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:24.764679 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4059bf-c8ac-4e0a-b33b-21e6a7adc278 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:24.764967 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4059bf-c8ac-4e0a-b33b-21e6a7adc278 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:24.765244 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4059bf-c8ac-4e0a-b33b-21e6a7adc278 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:24.765996 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4059bf-c8ac-4e0a-b33b-21e6a7adc278 None admin] RBAC: Adding query filter params (name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:24.766425 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4059bf-c8ac-4e0a-b33b-21e6a7adc278 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:24.767925 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4059bf-c8ac-4e0a-b33b-21e6a7adc278 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:24.777115 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 420/861] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 6 16:55:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:24.778071 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4059bf-c8ac-4e0a-b33b-21e6a7adc278 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:24.795942 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24816337-71ca-4a02-9c4d-bb6dce9b546e None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:24.831875 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c850b7c-9417-4563-b056-1d47bc0c343e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:24.832547 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c850b7c-9417-4563-b056-1d47bc0c343e None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:24.832837 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c850b7c-9417-4563-b056-1d47bc0c343e None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:24.833118 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c850b7c-9417-4563-b056-1d47bc0c343e None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:24.833834 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c850b7c-9417-4563-b056-1d47bc0c343e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:24.834147 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c850b7c-9417-4563-b056-1d47bc0c343e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:24.836119 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c850b7c-9417-4563-b056-1d47bc0c343e None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:24.838285 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:24.847639 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 442/862] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Jul 6 16:55:24 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:24.847639 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c850b7c-9417-4563-b056-1d47bc0c343e None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:24.910420 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4059bf-c8ac-4e0a-b33b-21e6a7adc278 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:24.920672 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:24.961218 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5ba4dc-6a8e-4ca7-b71e-477f57c8912c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:24.961900 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ba4dc-6a8e-4ca7-b71e-477f57c8912c None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:24.962183 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ba4dc-6a8e-4ca7-b71e-477f57c8912c None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:24.962461 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ba4dc-6a8e-4ca7-b71e-477f57c8912c None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:24.962649 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c850b7c-9417-4563-b056-1d47bc0c343e None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:24.963187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5ba4dc-6a8e-4ca7-b71e-477f57c8912c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:24.964928 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5ba4dc-6a8e-4ca7-b71e-477f57c8912c None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:24.999591 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a239d6-8d6d-49cd-9dcf-636663118852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:25.000492 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a239d6-8d6d-49cd-9dcf-636663118852 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:25.000773 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a239d6-8d6d-49cd-9dcf-636663118852 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:25.001053 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a239d6-8d6d-49cd-9dcf-636663118852 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:25.014463 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a239d6-8d6d-49cd-9dcf-636663118852 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:25.015902 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a239d6-8d6d-49cd-9dcf-636663118852 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:25.036052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-c8a239d6-8d6d-49cd-9dcf-636663118852 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:25.036052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:25.036052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:25.036052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:25.036052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:25.036052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:25.036052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:25.036052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:25.036052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:25.036052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:25.036052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:25.036052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:25.036052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:55:25.036052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:55:25.036052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:55:25.041735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:55:25.041735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:25.041735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:25.041735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:25.041735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:25.041735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:25.041735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:25.041735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:25.041735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:25.041735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:25.041735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:25.041735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:25.041735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:25.041735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:25.041735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:25.041735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:25.041735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:25.042757 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:55:25.042757 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:55:25.042757 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:25.042757 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:25.043066 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 421/863] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Jul 6 16:55:24 2022] POST /identity/v3/services => generated 251 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 16:55:25.043141 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 443/864] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:25.043217 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5ba4dc-6a8e-4ca7-b71e-477f57c8912c None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:25.171713 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5ba4dc-6a8e-4ca7-b71e-477f57c8912c None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:25.211580 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62281c6-5721-4a64-ae91-407861479438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:25.212289 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62281c6-5721-4a64-ae91-407861479438 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:25.212573 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62281c6-5721-4a64-ae91-407861479438 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:25.212844 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62281c6-5721-4a64-ae91-407861479438 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:25.213572 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62281c6-5721-4a64-ae91-407861479438 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:25.213877 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62281c6-5721-4a64-ae91-407861479438 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:25.215250 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62281c6-5721-4a64-ae91-407861479438 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:25.228274 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 422/865] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:55:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:25.233875 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a239d6-8d6d-49cd-9dcf-636663118852 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:25.291188 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:25.344458 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a239d6-8d6d-49cd-9dcf-636663118852 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:25.383356 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8eb3f7-2db7-49bd-96bc-58969eb59b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:25.384212 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8eb3f7-2db7-49bd-96bc-58969eb59b31 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:25.384500 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8eb3f7-2db7-49bd-96bc-58969eb59b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:25.384779 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8eb3f7-2db7-49bd-96bc-58969eb59b31 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:25.393350 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8eb3f7-2db7-49bd-96bc-58969eb59b31 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:25.394804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8eb3f7-2db7-49bd-96bc-58969eb59b31 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:25.409982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-ed8eb3f7-2db7-49bd-96bc-58969eb59b31 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 06 16:55:25.409982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:25.409982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:25.409982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:25.409982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:25.409982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:25.409982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:25.409982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:25.409982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:25.409982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:25.409982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:25.409982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:25.409982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 16:55:25.409982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 16:55:25.409982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 16:55:25.409982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 16:55:25.410945 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:25.410945 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:25.410945 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 16:55:25.410945 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:25.410945 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 16:55:25.410945 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:25.410945 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:25.410945 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:25.410945 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:25.410945 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:25.410945 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:25.410945 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:25.410945 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:25.410945 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:25.410945 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:25.410945 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:25.410945 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:25.410945 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:25.412382 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:25.412382 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:25.412382 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 16:55:25.412382 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 16:55:25.412382 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 16:55:25.412382 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 16:55:25.412382 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 16:55:25.412382 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 16:55:25.412382 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 06 16:55:25.412382 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:25.412382 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 444/866] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Jul 6 16:55:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:25.417102 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62281c6-5721-4a64-ae91-407861479438 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:25.443894 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:25.519624 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62281c6-5721-4a64-ae91-407861479438 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:25.558300 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f393d26-c0ae-4b89-94fc-4808ae6e6b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:25.558906 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f393d26-c0ae-4b89-94fc-4808ae6e6b6b None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:25.559180 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f393d26-c0ae-4b89-94fc-4808ae6e6b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:25.559459 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f393d26-c0ae-4b89-94fc-4808ae6e6b6b None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:25.560345 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f393d26-c0ae-4b89-94fc-4808ae6e6b6b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:25.560656 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f393d26-c0ae-4b89-94fc-4808ae6e6b6b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:25.562111 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f393d26-c0ae-4b89-94fc-4808ae6e6b6b None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:25.583197 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 423/867] 10.222.0.40 () {56 vars in 1149 bytes} [Wed Jul 6 16:55:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:25.588916 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8eb3f7-2db7-49bd-96bc-58969eb59b31 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:25.697117 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8eb3f7-2db7-49bd-96bc-58969eb59b31 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:25.707759 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:25.737177 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f1ffa6-4af3-43d1-a2cf-7f9b18cf6491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:25.737846 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f1ffa6-4af3-43d1-a2cf-7f9b18cf6491 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:25.738133 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f1ffa6-4af3-43d1-a2cf-7f9b18cf6491 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:25.738410 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f1ffa6-4af3-43d1-a2cf-7f9b18cf6491 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:25.809479 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f1ffa6-4af3-43d1-a2cf-7f9b18cf6491 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:25.810854 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f1ffa6-4af3-43d1-a2cf-7f9b18cf6491 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:25.915924 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 445/868] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:55:25 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 16:55:25.917861 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f393d26-c0ae-4b89-94fc-4808ae6e6b6b None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:26.005064 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:26.025545 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f393d26-c0ae-4b89-94fc-4808ae6e6b6b None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:26.068716 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fca28f-ec77-47be-9ffc-099d986ecb46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:26.069437 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fca28f-ec77-47be-9ffc-099d986ecb46 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:26.069780 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fca28f-ec77-47be-9ffc-099d986ecb46 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:26.070059 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fca28f-ec77-47be-9ffc-099d986ecb46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:26.080653 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fca28f-ec77-47be-9ffc-099d986ecb46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:26.082165 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fca28f-ec77-47be-9ffc-099d986ecb46 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:26.100573 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-47fca28f-ec77-47be-9ffc-099d986ecb46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:26.100573 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:26.100573 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:26.100573 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:26.100573 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:26.100573 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:26.100573 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:26.100573 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:26.100573 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:26.100573 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:26.100573 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:26.100573 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:26.100573 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:55:26.100573 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:55:26.100573 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:55:26.102054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:55:26.102054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:26.102054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:26.102054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:26.102054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:26.102054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:26.102054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:26.102054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:26.102054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:26.102054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:26.102054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:26.102054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:26.102054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:26.102054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:26.102054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:26.102054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:26.102054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:26.103962 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:55:26.103962 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:55:26.103962 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:26.103962 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:26.103962 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 424/869] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:55:26.108223 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f1ffa6-4af3-43d1-a2cf-7f9b18cf6491 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:26.120087 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:26.210774 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f1ffa6-4af3-43d1-a2cf-7f9b18cf6491 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:26.249687 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055b67ce-1dbb-431a-9390-df8c9ff79a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:26.250735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b67ce-1dbb-431a-9390-df8c9ff79a73 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:26.251225 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b67ce-1dbb-431a-9390-df8c9ff79a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:26.251744 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055b67ce-1dbb-431a-9390-df8c9ff79a73 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:26.252697 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055b67ce-1dbb-431a-9390-df8c9ff79a73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:26.253205 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055b67ce-1dbb-431a-9390-df8c9ff79a73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:26.254760 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055b67ce-1dbb-431a-9390-df8c9ff79a73 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:26.268216 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 446/870] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:55:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:26.277081 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fca28f-ec77-47be-9ffc-099d986ecb46 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:26.385993 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fca28f-ec77-47be-9ffc-099d986ecb46 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:26.421546 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ca1b5d-0bf0-4680-8360-225f61a50645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:26.422694 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ca1b5d-0bf0-4680-8360-225f61a50645 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:26.423221 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ca1b5d-0bf0-4680-8360-225f61a50645 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:26.423700 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ca1b5d-0bf0-4680-8360-225f61a50645 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:26.425879 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:26.432328 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ca1b5d-0bf0-4680-8360-225f61a50645 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:26.434024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ca1b5d-0bf0-4680-8360-225f61a50645 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:26.448632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-21ca1b5d-0bf0-4680-8360-225f61a50645 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:26.448632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:26.448632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:26.448632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:26.448632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:26.448632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:26.448632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:26.448632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:26.448632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:26.448632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:26.448632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:26.448632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:26.448632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 16:55:26.448632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 16:55:26.448632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 16:55:26.449548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 16:55:26.449548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:26.449548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:26.449548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:26.449548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:26.449548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:26.449548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:26.449548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:26.449548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:26.449548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:26.449548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:26.449548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:26.449548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:26.449548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:26.449548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:26.449548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:26.449548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:26.450411 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 16:55:26.450411 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 16:55:26.450411 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 16:55:26.450411 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 16:55:26.450411 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 16:55:26.450411 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 16:55:26.450411 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:26.450411 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:26.452954 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 425/871] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:55:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:26.460866 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055b67ce-1dbb-431a-9390-df8c9ff79a73 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:26.533671 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:26.570009 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055b67ce-1dbb-431a-9390-df8c9ff79a73 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:26.605373 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24534024-9401-40a4-aeaa-a661f14d860b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:26.606833 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24534024-9401-40a4-aeaa-a661f14d860b None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:26.607362 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24534024-9401-40a4-aeaa-a661f14d860b None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:26.607873 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24534024-9401-40a4-aeaa-a661f14d860b None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:26.608886 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24534024-9401-40a4-aeaa-a661f14d860b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:26.609396 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24534024-9401-40a4-aeaa-a661f14d860b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:26.611029 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24534024-9401-40a4-aeaa-a661f14d860b None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:26.616533 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9f50c-3963-4244-9fdf-77e8e2c3687f None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:26.617028 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9f50c-3963-4244-9fdf-77e8e2c3687f None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:26.617126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9f50c-3963-4244-9fdf-77e8e2c3687f None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:26.618723 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 426/872] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:26.623999 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2829ae6d-2be4-42c9-b61f-42c2eb2d5b1d None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:26.624301 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2829ae6d-2be4-42c9-b61f-42c2eb2d5b1d None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:26.624559 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2829ae6d-2be4-42c9-b61f-42c2eb2d5b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:26.625891 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 447/873] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 6 16:55:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:26.633246 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24534024-9401-40a4-aeaa-a661f14d860b None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:26.646380 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-2829ae6d-2be4-42c9-b61f-42c2eb2d5b1d None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:26.648708 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-2829ae6d-2be4-42c9-b61f-42c2eb2d5b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:26.739759 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 427/874] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:26 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:55:26.755848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24534024-9401-40a4-aeaa-a661f14d860b None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:26.758760 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2829ae6d-2be4-42c9-b61f-42c2eb2d5b1d None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:26.790552 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:26.797534 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65855f39-e011-4d97-a549-59d8a0fe2f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:26.798551 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65855f39-e011-4d97-a549-59d8a0fe2f9e None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:26.799216 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65855f39-e011-4d97-a549-59d8a0fe2f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:26.799776 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65855f39-e011-4d97-a549-59d8a0fe2f9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:26.803233 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65855f39-e011-4d97-a549-59d8a0fe2f9e None admin] RBAC: Adding query filter params (role.id=750bca9c32ac4aa49adcbee30463f1fa, scope.project.id=7938b0346d4240ab89774327d09a89ae, user.id=1905e6f8c3e24e3cbfefce2827baa370) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:26.803233 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65855f39-e011-4d97-a549-59d8a0fe2f9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:26.803233 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65855f39-e011-4d97-a549-59d8a0fe2f9e None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:26.814926 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 448/875] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:55:26 2022] GET /identity/v3/role_assignments?user.id=1905e6f8c3e24e3cbfefce2827baa370&scope.project.id=7938b0346d4240ab89774327d09a89ae&role.id=750bca9c32ac4aa49adcbee30463f1fa => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:26.881890 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2829ae6d-2be4-42c9-b61f-42c2eb2d5b1d None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:26.910719 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:26.923215 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b59591-f6d9-4ea9-bb3e-1d04c17359f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:26.929130 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b59591-f6d9-4ea9-bb3e-1d04c17359f6 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:26.929130 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b59591-f6d9-4ea9-bb3e-1d04c17359f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:26.929130 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b59591-f6d9-4ea9-bb3e-1d04c17359f6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:26.929130 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b59591-f6d9-4ea9-bb3e-1d04c17359f6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:26.929130 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b59591-f6d9-4ea9-bb3e-1d04c17359f6 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:26.940094 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-05b59591-f6d9-4ea9-bb3e-1d04c17359f6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 16:55:26.940094 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:26.940094 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 16:55:26.940094 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 16:55:26.940094 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 16:55:26.940094 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 16:55:26.940094 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 16:55:26.940094 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:26.940094 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 16:55:26.940094 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:26.940094 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 16:55:26.940094 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:26.940094 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:26.940094 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:26.940094 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:26.940094 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:26.940094 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:26.941452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:26.941452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:26.941452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:26.941452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:26.941452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:26.941452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:26.941452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 16:55:26.941452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 16:55:26.941452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 16:55:26.941452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 16:55:26.941452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:26.941452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:26.941452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:26.941452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:26.941452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:26.941452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:26.942385 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:26.942385 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:26.942385 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:26.942385 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:26.942385 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:26.942385 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:26.942385 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:26.942385 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:26.942385 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:26.942385 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:26.942385 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 16:55:26.942385 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:26.942385 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 16:55:26.942385 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:26.943129 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350630dc-be06-4afe-a729-7bb642ae3755 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:26.943129 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350630dc-be06-4afe-a729-7bb642ae3755 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:26.943129 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350630dc-be06-4afe-a729-7bb642ae3755 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:26.943324 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 428/876] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Jul 6 16:55:26 2022] GET /identity/v3/services/object-store => generated 93 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:55:26.943393 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 449/877] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 16:55:26.948549 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f7317-ff8e-4d45-b225-1322aa3fb687 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:26.948846 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f7317-ff8e-4d45-b225-1322aa3fb687 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:26.949068 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350630dc-be06-4afe-a729-7bb642ae3755 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:26.949203 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7f7317-ff8e-4d45-b225-1322aa3fb687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:26.971020 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-6c7f7317-ff8e-4d45-b225-1322aa3fb687 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:26.972243 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-6c7f7317-ff8e-4d45-b225-1322aa3fb687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:27.055096 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 429/878] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:26 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:55:27.072285 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f7317-ff8e-4d45-b225-1322aa3fb687 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:27.083973 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350630dc-be06-4afe-a729-7bb642ae3755 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:27.127987 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2352d54b-ed09-42c6-a2b1-4fd1849fb3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:27.128666 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352d54b-ed09-42c6-a2b1-4fd1849fb3e7 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:27.128956 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352d54b-ed09-42c6-a2b1-4fd1849fb3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:27.129239 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352d54b-ed09-42c6-a2b1-4fd1849fb3e7 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:27.129956 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2352d54b-ed09-42c6-a2b1-4fd1849fb3e7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:27.130338 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2352d54b-ed09-42c6-a2b1-4fd1849fb3e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:27.131713 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2352d54b-ed09-42c6-a2b1-4fd1849fb3e7 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:27.136775 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 450/879] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Jul 6 16:55:26 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:27.142862 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2352d54b-ed09-42c6-a2b1-4fd1849fb3e7 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:27.199876 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7f7317-ff8e-4d45-b225-1322aa3fb687 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:27.212870 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:27.218993 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:27.236736 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f70c1e-48d7-4e22-a226-5c3642097bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:27.237408 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f70c1e-48d7-4e22-a226-5c3642097bd1 None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:27.237692 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f70c1e-48d7-4e22-a226-5c3642097bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:27.237964 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f70c1e-48d7-4e22-a226-5c3642097bd1 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:27.238761 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f70c1e-48d7-4e22-a226-5c3642097bd1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:27.240160 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f70c1e-48d7-4e22-a226-5c3642097bd1 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:27.258124 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2352d54b-ed09-42c6-a2b1-4fd1849fb3e7 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:27.291740 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6f834c-c00c-4a9b-96b9-d3e510121f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:27.292424 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f834c-c00c-4a9b-96b9-d3e510121f0a None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:27.292712 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f834c-c00c-4a9b-96b9-d3e510121f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:27.292991 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6f834c-c00c-4a9b-96b9-d3e510121f0a None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:27.293701 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6f834c-c00c-4a9b-96b9-d3e510121f0a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:27.294013 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6f834c-c00c-4a9b-96b9-d3e510121f0a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:27.295372 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6f834c-c00c-4a9b-96b9-d3e510121f0a None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:27.299836 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 451/880] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Jul 6 16:55:27 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:27.305371 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69ca8d-3114-4d37-8c64-d30c9c9f8e5c None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:27.305671 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69ca8d-3114-4d37-8c64-d30c9c9f8e5c None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:27.305942 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b69ca8d-3114-4d37-8c64-d30c9c9f8e5c None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:27.307624 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 452/881] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:27.313738 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d56b0-a951-461b-a314-4324f035dbc2 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:27.314490 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d56b0-a951-461b-a314-4324f035dbc2 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:27.314936 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5d56b0-a951-461b-a314-4324f035dbc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:27.317812 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 430/882] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Jul 6 16:55:27 2022] POST /identity/v3/services => generated 269 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 16:55:27.340004 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-5f5d56b0-a951-461b-a314-4324f035dbc2 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:27.341771 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-5f5d56b0-a951-461b-a314-4324f035dbc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:27.437340 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 453/883] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:27 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:55:27.461350 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f70c1e-48d7-4e22-a226-5c3642097bd1 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:27.568288 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f70c1e-48d7-4e22-a226-5c3642097bd1 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:27.603072 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb04073-911f-4efb-a089-c21accff42d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:27.603912 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb04073-911f-4efb-a089-c21accff42d1 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:27.604201 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb04073-911f-4efb-a089-c21accff42d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:27.604479 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb04073-911f-4efb-a089-c21accff42d1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:27.605137 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb04073-911f-4efb-a089-c21accff42d1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:27.606522 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb04073-911f-4efb-a089-c21accff42d1 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:27.619060 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-9eb04073-911f-4efb-a089-c21accff42d1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 06 16:55:27.619060 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:27.619060 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 16:55:27.619060 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 16:55:27.619060 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 16:55:27.619060 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 16:55:27.619060 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 16:55:27.619060 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:27.619060 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 06 16:55:27.619060 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:27.619060 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 16:55:27.619060 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:27.619060 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:27.619060 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:27.619060 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:27.619060 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:27.619060 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:27.620527 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:27.620527 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:27.620527 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:27.620527 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:27.620527 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:27.620527 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:27.620527 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 16:55:27.620527 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 16:55:27.620527 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 16:55:27.620527 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 16:55:27.620527 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:27.620527 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:27.620527 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:27.620527 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:27.620527 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:27.620527 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:27.621330 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:27.621330 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:27.621330 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:27.621330 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:27.621330 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:27.621330 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:27.621330 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:27.621330 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:27.621330 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:27.621330 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:27.621330 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 16:55:27.621330 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:27.621330 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 06 16:55:27.621330 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:27.621330 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 431/884] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 6 16:55:27 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:27.626461 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d56b0-a951-461b-a314-4324f035dbc2 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:27.679242 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:27.733302 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5d56b0-a951-461b-a314-4324f035dbc2 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:27.770205 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac063e-fabf-418c-b400-57c8fcd84dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:27.771513 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac063e-fabf-418c-b400-57c8fcd84dbd None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:27.772034 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac063e-fabf-418c-b400-57c8fcd84dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:27.773078 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac063e-fabf-418c-b400-57c8fcd84dbd None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:27.773983 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bac063e-fabf-418c-b400-57c8fcd84dbd None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:27.774485 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bac063e-fabf-418c-b400-57c8fcd84dbd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:27.776066 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bac063e-fabf-418c-b400-57c8fcd84dbd None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:27.781469 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 454/885] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 6 16:55:27 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:27.791750 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb04073-911f-4efb-a089-c21accff42d1 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:27.831622 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:27.898688 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb04073-911f-4efb-a089-c21accff42d1 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:27.932905 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d0fab-50e7-4cd0-929c-533b1aa25888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:27.933583 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d0fab-50e7-4cd0-929c-533b1aa25888 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:27.933871 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d0fab-50e7-4cd0-929c-533b1aa25888 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:27.934153 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d0fab-50e7-4cd0-929c-533b1aa25888 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:27.934866 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d0fab-50e7-4cd0-929c-533b1aa25888 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:27.935175 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d0fab-50e7-4cd0-929c-533b1aa25888 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:27.936713 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d0fab-50e7-4cd0-929c-533b1aa25888 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:27.942299 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 432/886] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 6 16:55:27 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:27.948480 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac063e-fabf-418c-b400-57c8fcd84dbd None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:28.051327 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac063e-fabf-418c-b400-57c8fcd84dbd None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:28.065525 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:28.088372 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad1e5be-2ef6-425f-be97-c9e929fc716d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:28.089519 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1e5be-2ef6-425f-be97-c9e929fc716d None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:28.090008 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1e5be-2ef6-425f-be97-c9e929fc716d None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:28.090292 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1e5be-2ef6-425f-be97-c9e929fc716d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:28.091169 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad1e5be-2ef6-425f-be97-c9e929fc716d None admin] RBAC: Adding query filter params (interface=public, service_id=b82101b32dc649eca98d67f61c7f8478, region_id=RegionOne) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:28.091526 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad1e5be-2ef6-425f-be97-c9e929fc716d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:28.092953 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad1e5be-2ef6-425f-be97-c9e929fc716d None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:28.099835 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 455/887] 10.222.0.40 () {56 vars in 1251 bytes} [Wed Jul 6 16:55:27 2022] GET /identity/v3/endpoints?service_id=b82101b32dc649eca98d67f61c7f8478&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:28.106938 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d0fab-50e7-4cd0-929c-533b1aa25888 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:28.215264 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d0fab-50e7-4cd0-929c-533b1aa25888 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:28.222265 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:28.250545 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e083e6f4-0782-43b2-8146-fd375697984a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:28.251273 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083e6f4-0782-43b2-8146-fd375697984a None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:28.251561 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083e6f4-0782-43b2-8146-fd375697984a None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:28.251926 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083e6f4-0782-43b2-8146-fd375697984a None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:28.252661 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e083e6f4-0782-43b2-8146-fd375697984a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:28.254098 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e083e6f4-0782-43b2-8146-fd375697984a None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:28.260204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 433/888] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Jul 6 16:55:28 2022] GET /identity/v3/services => generated 1268 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 16:55:29.054205 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72bd4d2-04bf-4cbf-89d1-38c0fe5dbc48 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:29.054704 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72bd4d2-04bf-4cbf-89d1-38c0fe5dbc48 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:29.054704 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72bd4d2-04bf-4cbf-89d1-38c0fe5dbc48 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:29.056337 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 456/889] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:29.061626 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc17fc-5217-42c5-9f56-fd2c25af9691 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:29.062022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc17fc-5217-42c5-9f56-fd2c25af9691 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:29.062187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc17fc-5217-42c5-9f56-fd2c25af9691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:29.079892 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-43fc17fc-5217-42c5-9f56-fd2c25af9691 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:29.081066 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-43fc17fc-5217-42c5-9f56-fd2c25af9691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:29.149379 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 434/890] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:29 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 16:55:29.172410 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72bd4d2-04bf-4cbf-89d1-38c0fe5dbc48 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:29.277140 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72bd4d2-04bf-4cbf-89d1-38c0fe5dbc48 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:29.316438 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3680e4b-d22c-4b93-bf27-7e10f10cbe5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:29.317185 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3680e4b-d22c-4b93-bf27-7e10f10cbe5c None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:29.317466 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3680e4b-d22c-4b93-bf27-7e10f10cbe5c None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:29.317744 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3680e4b-d22c-4b93-bf27-7e10f10cbe5c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:29.329062 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3680e4b-d22c-4b93-bf27-7e10f10cbe5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:55:29.337208 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-d3680e4b-d22c-4b93-bf27-7e10f10cbe5c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 06 16:55:29.337208 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:29.337208 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:29.337208 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:29.337208 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:29.337208 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:29.337208 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:29.337208 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:29.337208 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:29.337208 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:29.337208 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:29.337208 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:29.337208 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:55:29.337208 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:55:29.337208 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:55:29.337208 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:55:29.338117 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:55:29.338117 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:55:29.338117 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:55:29.338117 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:55:29.338117 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:29.338117 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:29.338117 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:29.338117 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:29.338117 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:29.338117 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:29.338117 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:29.338117 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:29.338117 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:29.338117 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:29.338117 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:29.338117 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:29.338952 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:29.338952 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:29.338952 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:29.338952 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:29.338952 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:55:29.338952 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:55:29.338952 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:55:29.338952 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:55:29.338952 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:55:29.338952 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:55:29.338952 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 06 16:55:29.338952 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:29.338952 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 457/891] 10.222.0.40 () {56 vars in 1120 bytes} [Wed Jul 6 16:55:29 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:29.344362 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc17fc-5217-42c5-9f56-fd2c25af9691 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:29.355623 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3bd7da-4c7a-42a7-989d-a674a87a50c0 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:29.356014 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3bd7da-4c7a-42a7-989d-a674a87a50c0 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:29.356366 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3bd7da-4c7a-42a7-989d-a674a87a50c0 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:29.358047 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 458/892] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 16:55:29.362913 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b380e-2801-4896-a59a-269fe960e24f None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:29.363207 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b380e-2801-4896-a59a-269fe960e24f None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:29.363487 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b380e-2801-4896-a59a-269fe960e24f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:29.380739 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-c96b380e-2801-4896-a59a-269fe960e24f None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:29.381968 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c96b380e-2801-4896-a59a-269fe960e24f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:29.431358 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:29.459225 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:29.462854 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc17fc-5217-42c5-9f56-fd2c25af9691 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:29.472326 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 459/893] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:29 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 16:55:29.489602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b9d662-c3c7-4af2-9ebd-4ac8d4d4e7fa None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:29.490127 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b9d662-c3c7-4af2-9ebd-4ac8d4d4e7fa None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:29.490553 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b9d662-c3c7-4af2-9ebd-4ac8d4d4e7fa None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:29.492907 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 460/894] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 16:55:29.496610 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b9d662-c3c7-4af2-9ebd-4ac8d4d4e7fa None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:29.502536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f950cc98-7683-4c6b-8116-d9b2a9e760c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:29.503210 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950cc98-7683-4c6b-8116-d9b2a9e760c8 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:29.503492 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950cc98-7683-4c6b-8116-d9b2a9e760c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:29.503787 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950cc98-7683-4c6b-8116-d9b2a9e760c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:29.504542 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f950cc98-7683-4c6b-8116-d9b2a9e760c8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:29.504847 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f950cc98-7683-4c6b-8116-d9b2a9e760c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:29.506204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f950cc98-7683-4c6b-8116-d9b2a9e760c8 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:29.515720 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 435/895] 10.222.0.40 () {56 vars in 1115 bytes} [Wed Jul 6 16:55:29 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:29.516920 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4b1e8-8379-4a23-b3ca-95313a34df5d None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:29.517207 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4b1e8-8379-4a23-b3ca-95313a34df5d None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:29.517483 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e4b1e8-8379-4a23-b3ca-95313a34df5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:29.533761 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-93e4b1e8-8379-4a23-b3ca-95313a34df5d None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:29.534892 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-93e4b1e8-8379-4a23-b3ca-95313a34df5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:29.604120 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 436/896] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:29 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:55:29.605465 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e4b1e8-8379-4a23-b3ca-95313a34df5d None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:29.612243 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b9d662-c3c7-4af2-9ebd-4ac8d4d4e7fa None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:29.648149 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242889c8-327f-4647-937c-b7f347f3710d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:29.648920 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242889c8-327f-4647-937c-b7f347f3710d None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:29.649203 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242889c8-327f-4647-937c-b7f347f3710d None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:29.649476 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242889c8-327f-4647-937c-b7f347f3710d None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:29.650188 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242889c8-327f-4647-937c-b7f347f3710d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:29.651550 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242889c8-327f-4647-937c-b7f347f3710d None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:29.703000 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:29.711220 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e4b1e8-8379-4a23-b3ca-95313a34df5d None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:29.724141 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 461/897] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Jul 6 16:55:29 2022] POST /identity/v3/services => generated 241 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 16:55:29.724141 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242889c8-327f-4647-937c-b7f347f3710d None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:29.771006 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a110761-dadb-4732-953b-e1a00d17d325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:29.772158 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a110761-dadb-4732-953b-e1a00d17d325 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:29.772990 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a110761-dadb-4732-953b-e1a00d17d325 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:29.773269 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a110761-dadb-4732-953b-e1a00d17d325 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:29.785126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a110761-dadb-4732-953b-e1a00d17d325 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:29.786616 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a110761-dadb-4732-953b-e1a00d17d325 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:29.786963 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:29.807508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-0a110761-dadb-4732-953b-e1a00d17d325 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:29.807508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:29.807508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:29.807508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:29.807508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:29.807508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:29.807508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:29.807508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:29.807508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:29.807508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:29.807508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:29.807508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:29.807508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:55:29.807508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:55:29.807508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:55:29.808897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:55:29.808897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:29.808897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:29.808897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:29.808897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:29.808897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:29.808897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:29.808897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:29.808897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:29.808897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:29.808897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:29.808897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:29.808897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:29.808897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:29.808897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:29.808897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:29.808897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:29.810164 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:55:29.810164 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:55:29.810164 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:29.810164 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:29.810810 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 437/898] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:29.820839 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a110761-dadb-4732-953b-e1a00d17d325 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:29.878942 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242889c8-327f-4647-937c-b7f347f3710d None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:29.924025 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ccd36a-f7db-4bd2-8012-19e84a390238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:29.924853 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ccd36a-f7db-4bd2-8012-19e84a390238 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:29.925142 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ccd36a-f7db-4bd2-8012-19e84a390238 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:29.925417 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ccd36a-f7db-4bd2-8012-19e84a390238 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:29.926084 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ccd36a-f7db-4bd2-8012-19e84a390238 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:29.927468 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ccd36a-f7db-4bd2-8012-19e84a390238 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:29.940243 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-87ccd36a-f7db-4bd2-8012-19e84a390238 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 06 16:55:29.940243 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:29.940243 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 16:55:29.940243 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 16:55:29.940243 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 16:55:29.940243 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 16:55:29.940243 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 16:55:29.940243 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:29.940243 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 06 16:55:29.940243 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:29.940243 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 16:55:29.940243 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:29.940243 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:29.940243 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:29.940243 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:29.940243 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:29.941245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:29.941245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:29.941245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:29.941245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:29.941245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:29.941245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:29.941245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:29.941245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 16:55:29.941245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 16:55:29.941245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 16:55:29.941245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 16:55:29.941245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:29.941245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:29.941245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:29.941245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:29.941245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:29.942515 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:29.942515 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:29.942515 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:29.942515 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:29.942515 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:29.942515 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:29.942515 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:29.942515 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:29.942515 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:29.942515 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:29.942515 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:29.942515 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 16:55:29.942515 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:29.942515 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 06 16:55:29.942515 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:29.942515 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 462/899] 10.222.0.40 () {56 vars in 1132 bytes} [Wed Jul 6 16:55:29 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:29.948570 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ccd36a-f7db-4bd2-8012-19e84a390238 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:29.952331 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a110761-dadb-4732-953b-e1a00d17d325 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:30.001543 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824a65e-bda8-47b4-9961-0b56fd4525dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:30.002467 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824a65e-bda8-47b4-9961-0b56fd4525dd None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:30.003144 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824a65e-bda8-47b4-9961-0b56fd4525dd None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:30.003431 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824a65e-bda8-47b4-9961-0b56fd4525dd None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:30.004212 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6824a65e-bda8-47b4-9961-0b56fd4525dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:30.004523 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6824a65e-bda8-47b4-9961-0b56fd4525dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:30.006440 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6824a65e-bda8-47b4-9961-0b56fd4525dd None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:30.018366 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:30.024579 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 438/900] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:55:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:30.030230 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824a65e-bda8-47b4-9961-0b56fd4525dd None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:30.055484 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:30.071904 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ccd36a-f7db-4bd2-8012-19e84a390238 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:30.118504 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f11d8-b870-4651-90b3-4ceb662558ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:30.119239 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f11d8-b870-4651-90b3-4ceb662558ff None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:30.119532 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f11d8-b870-4651-90b3-4ceb662558ff None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:30.119842 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8f11d8-b870-4651-90b3-4ceb662558ff None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:30.120560 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f11d8-b870-4651-90b3-4ceb662558ff None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:30.120872 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f11d8-b870-4651-90b3-4ceb662558ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:30.122248 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8f11d8-b870-4651-90b3-4ceb662558ff None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:30.132493 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 463/901] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Jul 6 16:55:29 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:30.133825 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f11d8-b870-4651-90b3-4ceb662558ff None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:30.145096 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824a65e-bda8-47b4-9961-0b56fd4525dd None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:30.180889 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e0a55-6065-453b-8744-4b871de060d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:30.181735 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e0a55-6065-453b-8744-4b871de060d6 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:30.182019 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e0a55-6065-453b-8744-4b871de060d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:30.182294 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e0a55-6065-453b-8744-4b871de060d6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:30.190200 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e0a55-6065-453b-8744-4b871de060d6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:30.191653 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678e0a55-6065-453b-8744-4b871de060d6 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:30.207534 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-678e0a55-6065-453b-8744-4b871de060d6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 06 16:55:30.207534 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:30.207534 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:30.207534 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:30.207534 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:30.207534 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:30.207534 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:30.207534 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:30.207534 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:30.207534 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:30.207534 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:30.207534 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:30.207534 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 16:55:30.207534 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 16:55:30.207534 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 16:55:30.207534 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 16:55:30.209705 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:30.209705 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:30.209705 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 16:55:30.209705 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:30.209705 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 16:55:30.209705 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:30.209705 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:30.209705 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:30.209705 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:30.209705 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:30.209705 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:30.209705 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:30.209705 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:30.209705 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:30.209705 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:30.209705 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:30.209705 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:30.209705 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:30.212507 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:30.212507 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:30.212507 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 16:55:30.212507 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 16:55:30.212507 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 16:55:30.212507 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 16:55:30.212507 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 16:55:30.212507 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 16:55:30.212507 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 06 16:55:30.212507 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:30.219750 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 439/902] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Jul 6 16:55:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:55:30.219750 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e0a55-6065-453b-8744-4b871de060d6 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:30.242284 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8f11d8-b870-4651-90b3-4ceb662558ff None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:30.281351 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:30.282581 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91924e66-8010-408b-b6ee-6d9f420d5694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:30.283380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91924e66-8010-408b-b6ee-6d9f420d5694 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:30.283674 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91924e66-8010-408b-b6ee-6d9f420d5694 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:30.284177 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91924e66-8010-408b-b6ee-6d9f420d5694 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:30.285212 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91924e66-8010-408b-b6ee-6d9f420d5694 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:30.285650 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91924e66-8010-408b-b6ee-6d9f420d5694 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:30.287154 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91924e66-8010-408b-b6ee-6d9f420d5694 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:30.299888 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 464/903] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Jul 6 16:55:30 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:30.300796 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91924e66-8010-408b-b6ee-6d9f420d5694 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:30.326321 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:30.326584 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e0a55-6065-453b-8744-4b871de060d6 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:30.363059 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae389808-9897-45dd-b501-cd59288500aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:30.363758 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae389808-9897-45dd-b501-cd59288500aa None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:30.364053 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae389808-9897-45dd-b501-cd59288500aa None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:30.364332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae389808-9897-45dd-b501-cd59288500aa None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:30.365166 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae389808-9897-45dd-b501-cd59288500aa None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:30.365475 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae389808-9897-45dd-b501-cd59288500aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:30.366948 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae389808-9897-45dd-b501-cd59288500aa None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:30.390306 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 440/904] 10.222.0.40 () {56 vars in 1149 bytes} [Wed Jul 6 16:55:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:30.391497 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41778270-9efa-4fac-970b-efe56330f98b None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:30.392074 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41778270-9efa-4fac-970b-efe56330f98b None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:30.392383 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41778270-9efa-4fac-970b-efe56330f98b None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:30.396363 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 441/905] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:30.396363 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41778270-9efa-4fac-970b-efe56330f98b None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:30.419455 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91924e66-8010-408b-b6ee-6d9f420d5694 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:30.453602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b2900f-c54e-4600-85ce-6df91a924b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:30.454266 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2900f-c54e-4600-85ce-6df91a924b3e None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:30.454554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2900f-c54e-4600-85ce-6df91a924b3e None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:30.454829 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2900f-c54e-4600-85ce-6df91a924b3e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:30.455620 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b2900f-c54e-4600-85ce-6df91a924b3e None admin] RBAC: Adding query filter params (interface=public, service_id=7d248ab4aa82469f9d0624d382944506, region_id=RegionOne) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:30.455964 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b2900f-c54e-4600-85ce-6df91a924b3e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:30.457372 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b2900f-c54e-4600-85ce-6df91a924b3e None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:30.462425 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 465/906] 10.222.0.40 () {56 vars in 1251 bytes} [Wed Jul 6 16:55:30 2022] GET /identity/v3/endpoints?service_id=7d248ab4aa82469f9d0624d382944506&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:30.463621 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be99b409-3ffe-4a50-b579-7380aec156bb None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:30.463936 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be99b409-3ffe-4a50-b579-7380aec156bb None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:30.464216 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be99b409-3ffe-4a50-b579-7380aec156bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:30.480630 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-be99b409-3ffe-4a50-b579-7380aec156bb None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:30.481821 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-be99b409-3ffe-4a50-b579-7380aec156bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:30.507296 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41778270-9efa-4fac-970b-efe56330f98b None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:30.526750 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:30.545325 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c185a7-72de-481d-8461-1b09de6dff1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:30.546086 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c185a7-72de-481d-8461-1b09de6dff1b None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:30.546362 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c185a7-72de-481d-8461-1b09de6dff1b None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:30.546669 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c185a7-72de-481d-8461-1b09de6dff1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:30.550465 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 466/907] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:55:30.551354 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be99b409-3ffe-4a50-b579-7380aec156bb None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:30.569349 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:30.625967 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c185a7-72de-481d-8461-1b09de6dff1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:30.627419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c185a7-72de-481d-8461-1b09de6dff1b None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:30.666994 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be99b409-3ffe-4a50-b579-7380aec156bb None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:30.702806 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f1af0-60b1-45ac-9063-3f4eac82d2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:30.703472 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f1af0-60b1-45ac-9063-3f4eac82d2ef None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:30.703767 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f1af0-60b1-45ac-9063-3f4eac82d2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:30.704051 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593f1af0-60b1-45ac-9063-3f4eac82d2ef None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:30.704731 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593f1af0-60b1-45ac-9063-3f4eac82d2ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:30.706338 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593f1af0-60b1-45ac-9063-3f4eac82d2ef None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:30.719766 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 467/908] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Jul 6 16:55:30 2022] GET /identity/v3/services => generated 1497 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 16:55:30.719766 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f1af0-60b1-45ac-9063-3f4eac82d2ef None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:30.737937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 442/909] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:55:30 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 16:55:30.744296 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c185a7-72de-481d-8461-1b09de6dff1b None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:30.764647 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:30.832987 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:30.865875 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593f1af0-60b1-45ac-9063-3f4eac82d2ef None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:30.871915 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c185a7-72de-481d-8461-1b09de6dff1b None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:30.917249 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46f6d9d-0f15-47b0-be9c-97157c67fd49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:30.917977 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f6d9d-0f15-47b0-be9c-97157c67fd49 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:30.918257 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f6d9d-0f15-47b0-be9c-97157c67fd49 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:30.918530 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f6d9d-0f15-47b0-be9c-97157c67fd49 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:30.919189 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46f6d9d-0f15-47b0-be9c-97157c67fd49 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:30.920587 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46f6d9d-0f15-47b0-be9c-97157c67fd49 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:30.927042 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b064084-2d9b-41b0-8b6b-35241864459c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:30.927940 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b064084-2d9b-41b0-8b6b-35241864459c None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:30.928235 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b064084-2d9b-41b0-8b6b-35241864459c None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:30.928519 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b064084-2d9b-41b0-8b6b-35241864459c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:30.933347 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-c46f6d9d-0f15-47b0-be9c-97157c67fd49 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 06 16:55:30.933347 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:30.933347 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 16:55:30.933347 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 16:55:30.933347 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 16:55:30.933347 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 16:55:30.933347 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 16:55:30.933347 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:30.933347 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 06 16:55:30.933347 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:30.933347 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 16:55:30.933347 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:30.933347 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:30.933347 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:30.933347 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:30.933347 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:30.933347 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:30.934395 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:30.934395 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:30.934395 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:30.934395 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:30.934395 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:30.934395 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:30.934395 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 16:55:30.934395 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 16:55:30.934395 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 16:55:30.934395 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 16:55:30.934395 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:30.934395 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:30.934395 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:30.934395 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:30.934395 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:30.934395 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:30.935474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:30.935474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:30.935474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:30.935474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:30.935474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:30.935474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:30.935474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:30.935474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:30.935474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:30.935474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:30.935474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 16:55:30.935474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:30.935474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 06 16:55:30.935474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:30.935474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 468/910] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 6 16:55:30 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:30.942325 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b064084-2d9b-41b0-8b6b-35241864459c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:30.944797 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b064084-2d9b-41b0-8b6b-35241864459c None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:30.945522 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46f6d9d-0f15-47b0-be9c-97157c67fd49 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:30.963994 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-6b064084-2d9b-41b0-8b6b-35241864459c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:30.963994 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:30.963994 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:30.963994 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:30.963994 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:30.963994 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:30.963994 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:30.963994 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:30.963994 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:30.963994 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:30.963994 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:30.963994 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:30.963994 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:55:30.963994 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:55:30.963994 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:55:30.964937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:55:30.964937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:30.964937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:30.964937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:30.964937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:30.964937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:30.964937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:30.964937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:30.964937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:30.964937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:30.964937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:30.964937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:30.964937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:30.964937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:30.964937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:30.964937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:30.964937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:30.965820 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:55:30.965820 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:55:30.965820 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:30.965820 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:30.965820 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 443/911] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:55:30.970593 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b064084-2d9b-41b0-8b6b-35241864459c None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:31.074075 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46f6d9d-0f15-47b0-be9c-97157c67fd49 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:31.089221 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b064084-2d9b-41b0-8b6b-35241864459c None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:31.092017 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:31.119856 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28d0e58-6073-42c5-8786-75c4c3333470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:31.120646 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d0e58-6073-42c5-8786-75c4c3333470 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:31.121171 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d0e58-6073-42c5-8786-75c4c3333470 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:31.121702 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d0e58-6073-42c5-8786-75c4c3333470 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:31.122998 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28d0e58-6073-42c5-8786-75c4c3333470 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:31.123570 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28d0e58-6073-42c5-8786-75c4c3333470 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:31.126082 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28d0e58-6073-42c5-8786-75c4c3333470 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:31.132447 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 469/912] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 6 16:55:30 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:31.137822 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28d0e58-6073-42c5-8786-75c4c3333470 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:31.144208 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1ad5c1-7309-4ad9-9689-73d78bcc38ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:31.145205 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ad5c1-7309-4ad9-9689-73d78bcc38ec None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:31.146387 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ad5c1-7309-4ad9-9689-73d78bcc38ec None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:31.146894 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1ad5c1-7309-4ad9-9689-73d78bcc38ec None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:31.148050 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:31.149002 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1ad5c1-7309-4ad9-9689-73d78bcc38ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:31.149524 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1ad5c1-7309-4ad9-9689-73d78bcc38ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:31.151905 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1ad5c1-7309-4ad9-9689-73d78bcc38ec None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:31.173569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 444/913] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:55:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:31.175855 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1ad5c1-7309-4ad9-9689-73d78bcc38ec None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:31.252849 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28d0e58-6073-42c5-8786-75c4c3333470 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:31.298927 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdd7951-7dee-40d0-b481-9a8cea7635fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:31.299608 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd7951-7dee-40d0-b481-9a8cea7635fe None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:31.299929 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd7951-7dee-40d0-b481-9a8cea7635fe None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:31.300216 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdd7951-7dee-40d0-b481-9a8cea7635fe None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:31.300683 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1ad5c1-7309-4ad9-9689-73d78bcc38ec None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:31.300955 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdd7951-7dee-40d0-b481-9a8cea7635fe None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:31.301267 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdd7951-7dee-40d0-b481-9a8cea7635fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:31.302898 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdd7951-7dee-40d0-b481-9a8cea7635fe None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:31.314268 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 470/914] 10.222.0.40 () {56 vars in 1114 bytes} [Wed Jul 6 16:55:31 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:31.315865 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdd7951-7dee-40d0-b481-9a8cea7635fe None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:31.339086 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:31.347187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad492256-d7c9-4403-8548-299e45f2a6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:31.348078 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad492256-d7c9-4403-8548-299e45f2a6a6 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:31.348366 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad492256-d7c9-4403-8548-299e45f2a6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:31.348643 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad492256-d7c9-4403-8548-299e45f2a6a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:31.357519 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad492256-d7c9-4403-8548-299e45f2a6a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:31.358966 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad492256-d7c9-4403-8548-299e45f2a6a6 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:31.378488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-ad492256-d7c9-4403-8548-299e45f2a6a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:31.378488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:31.378488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:31.378488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:31.378488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:31.378488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:31.378488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:31.378488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:31.378488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:31.378488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:31.378488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:31.378488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:31.378488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 16:55:31.378488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 16:55:31.378488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 16:55:31.379414 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 16:55:31.379414 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:31.379414 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:31.379414 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:31.379414 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:31.379414 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:31.379414 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:31.379414 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:31.379414 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:31.379414 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:31.379414 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:31.379414 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:31.379414 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:31.379414 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:31.379414 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:31.379414 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:31.379414 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:31.380884 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 16:55:31.380884 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 16:55:31.380884 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 16:55:31.380884 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 16:55:31.380884 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 16:55:31.380884 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 16:55:31.380884 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:31.380884 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:31.383874 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 445/915] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:55:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:31.386331 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad492256-d7c9-4403-8548-299e45f2a6a6 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:31.394676 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:31.433387 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdd7951-7dee-40d0-b481-9a8cea7635fe None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:31.472052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee23823-a34c-4e55-b43d-5675bf607c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:31.473096 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee23823-a34c-4e55-b43d-5675bf607c37 None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:31.473616 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee23823-a34c-4e55-b43d-5675bf607c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:31.474095 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee23823-a34c-4e55-b43d-5675bf607c37 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:31.475175 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee23823-a34c-4e55-b43d-5675bf607c37 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:31.476935 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee23823-a34c-4e55-b43d-5675bf607c37 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:31.513344 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad492256-d7c9-4403-8548-299e45f2a6a6 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:31.548861 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd770d08-e028-46d8-afe6-9db428d923ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:31.549531 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd770d08-e028-46d8-afe6-9db428d923ca None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:31.549817 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd770d08-e028-46d8-afe6-9db428d923ca None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:31.550091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd770d08-e028-46d8-afe6-9db428d923ca None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:31.550882 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd770d08-e028-46d8-afe6-9db428d923ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:31.551194 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd770d08-e028-46d8-afe6-9db428d923ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:31.552657 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd770d08-e028-46d8-afe6-9db428d923ca None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:31.575790 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 446/917] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 6 16:55:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:31.575790 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd770d08-e028-46d8-afe6-9db428d923ca None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:31.576297 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 471/917] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 6 16:55:31 2022] POST /identity/v3/endpoints => generated 346 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 16:55:31.632654 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:31.689688 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd770d08-e028-46d8-afe6-9db428d923ca None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:31.734481 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9a0c29-dbe7-4107-a566-894c2ed362b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:31.735690 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a0c29-dbe7-4107-a566-894c2ed362b4 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:31.736214 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a0c29-dbe7-4107-a566-894c2ed362b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:31.736696 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9a0c29-dbe7-4107-a566-894c2ed362b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:31.738340 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a0c29-dbe7-4107-a566-894c2ed362b4 None admin] RBAC: Adding query filter params (role.id=4d3a77c1547c43aba11b60377d04d537, scope.project.id=7938b0346d4240ab89774327d09a89ae, user.id=1905e6f8c3e24e3cbfefce2827baa370) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:31.738893 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a0c29-dbe7-4107-a566-894c2ed362b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:31.740597 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9a0c29-dbe7-4107-a566-894c2ed362b4 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:31.747629 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 447/918] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:55:31 2022] GET /identity/v3/role_assignments?user.id=1905e6f8c3e24e3cbfefce2827baa370&scope.project.id=7938b0346d4240ab89774327d09a89ae&role.id=4d3a77c1547c43aba11b60377d04d537 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:31.882987 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23570c-cb5a-4e18-9c74-d814ee87d96e None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:31.883390 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23570c-cb5a-4e18-9c74-d814ee87d96e None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:31.883567 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df23570c-cb5a-4e18-9c74-d814ee87d96e None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:31.885240 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 472/919] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:31.890842 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d241876-ebd0-4430-8f2f-b3349553e4b6 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:31.891483 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d241876-ebd0-4430-8f2f-b3349553e4b6 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:31.892006 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d241876-ebd0-4430-8f2f-b3349553e4b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:31.910498 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-0d241876-ebd0-4430-8f2f-b3349553e4b6 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:31.912079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-0d241876-ebd0-4430-8f2f-b3349553e4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:31.990090 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 448/920] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:55:32.011956 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df23570c-cb5a-4e18-9c74-d814ee87d96e None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:32.124185 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df23570c-cb5a-4e18-9c74-d814ee87d96e None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:32.150427 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:32.161668 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c12b72-12b8-4d1c-941d-3e5244ec2613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:32.162475 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c12b72-12b8-4d1c-941d-3e5244ec2613 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:32.163955 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c12b72-12b8-4d1c-941d-3e5244ec2613 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:32.163955 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c12b72-12b8-4d1c-941d-3e5244ec2613 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:32.163955 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c12b72-12b8-4d1c-941d-3e5244ec2613 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:32.165144 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c12b72-12b8-4d1c-941d-3e5244ec2613 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:32.177178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-97c12b72-12b8-4d1c-941d-3e5244ec2613 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 16:55:32.177178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:32.177178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 16:55:32.177178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 16:55:32.177178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 16:55:32.177178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 16:55:32.177178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 16:55:32.177178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:32.177178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 16:55:32.177178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:32.177178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 16:55:32.177178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:32.177178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:32.177178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:32.177178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:32.177178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:32.177178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:32.178161 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:32.178161 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:32.178161 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:32.178161 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:32.178161 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:32.178161 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:32.178161 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 16:55:32.178161 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 16:55:32.178161 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 16:55:32.178161 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 16:55:32.178161 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:32.178161 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:32.178161 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:32.178161 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:32.178161 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:32.178161 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:32.179957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:32.179957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:32.179957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:32.179957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:32.179957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:32.179957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:32.179957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:32.179957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:32.179957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:32.179957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:32.179957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 16:55:32.179957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:32.179957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 16:55:32.179957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:32.179957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 473/921] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Jul 6 16:55:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:32.188833 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d241876-ebd0-4430-8f2f-b3349553e4b6 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:32.231958 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:32.291566 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d241876-ebd0-4430-8f2f-b3349553e4b6 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:32.326761 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a91d6dd-02ef-4d5f-b2a6-1632a3ef062e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:32.327435 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a91d6dd-02ef-4d5f-b2a6-1632a3ef062e None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:32.327737 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a91d6dd-02ef-4d5f-b2a6-1632a3ef062e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:32.328036 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a91d6dd-02ef-4d5f-b2a6-1632a3ef062e None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:32.328799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a91d6dd-02ef-4d5f-b2a6-1632a3ef062e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:32.329113 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a91d6dd-02ef-4d5f-b2a6-1632a3ef062e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:32.330493 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a91d6dd-02ef-4d5f-b2a6-1632a3ef062e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:32.336286 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 449/922] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Jul 6 16:55:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:32.341760 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c12b72-12b8-4d1c-941d-3e5244ec2613 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:32.447372 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c12b72-12b8-4d1c-941d-3e5244ec2613 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:32.482532 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e1ce8-f4ee-4a90-83b8-6d0386a1258e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:32.483289 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e1ce8-f4ee-4a90-83b8-6d0386a1258e None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:32.483577 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e1ce8-f4ee-4a90-83b8-6d0386a1258e None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:32.483926 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520e1ce8-f4ee-4a90-83b8-6d0386a1258e None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:32.484684 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520e1ce8-f4ee-4a90-83b8-6d0386a1258e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:32.484996 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520e1ce8-f4ee-4a90-83b8-6d0386a1258e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:32.486358 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520e1ce8-f4ee-4a90-83b8-6d0386a1258e None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:32.491900 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 474/923] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Jul 6 16:55:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:32.497544 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a91d6dd-02ef-4d5f-b2a6-1632a3ef062e None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:32.597221 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a91d6dd-02ef-4d5f-b2a6-1632a3ef062e None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:32.600025 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:32.630793 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4503fb0f-7b16-415e-9d22-e3c127c6b173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:32.632326 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4503fb0f-7b16-415e-9d22-e3c127c6b173 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:32.632820 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4503fb0f-7b16-415e-9d22-e3c127c6b173 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:32.633316 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4503fb0f-7b16-415e-9d22-e3c127c6b173 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:32.634386 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4503fb0f-7b16-415e-9d22-e3c127c6b173 None admin] RBAC: Adding query filter params (interface=public, service_id=ee7f1a7143e54e258d1e65d1b0c62a09, region_id=RegionOne) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:32.634895 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4503fb0f-7b16-415e-9d22-e3c127c6b173 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:32.636461 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4503fb0f-7b16-415e-9d22-e3c127c6b173 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:32.644484 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 450/924] 10.222.0.40 () {56 vars in 1251 bytes} [Wed Jul 6 16:55:32 2022] GET /identity/v3/endpoints?service_id=ee7f1a7143e54e258d1e65d1b0c62a09&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:32.651098 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e1ce8-f4ee-4a90-83b8-6d0386a1258e None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:32.704977 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:32.756073 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520e1ce8-f4ee-4a90-83b8-6d0386a1258e None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:32.792863 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5aca2f-9fd0-4537-ad47-706576f739a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:32.793538 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5aca2f-9fd0-4537-ad47-706576f739a2 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:32.793832 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5aca2f-9fd0-4537-ad47-706576f739a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:32.794110 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5aca2f-9fd0-4537-ad47-706576f739a2 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:32.794805 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5aca2f-9fd0-4537-ad47-706576f739a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:32.796202 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5aca2f-9fd0-4537-ad47-706576f739a2 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:32.802789 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 475/925] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Jul 6 16:55:32 2022] GET /identity/v3/services => generated 1497 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 16:55:32.910105 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2d8ec-2cd9-4237-8b71-1424c44c9a20 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:32.910463 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2d8ec-2cd9-4237-8b71-1424c44c9a20 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:32.910710 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b2d8ec-2cd9-4237-8b71-1424c44c9a20 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:32.912595 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 451/926] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:32.918646 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0f86a-59db-4ca1-ab2d-25e5439ffb0d None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:32.918983 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0f86a-59db-4ca1-ab2d-25e5439ffb0d None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:32.919242 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0f86a-59db-4ca1-ab2d-25e5439ffb0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:32.938593 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-f7c0f86a-59db-4ca1-ab2d-25e5439ffb0d None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:32.939864 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f7c0f86a-59db-4ca1-ab2d-25e5439ffb0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:33.019427 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 476/927] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:55:33.043695 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b2d8ec-2cd9-4237-8b71-1424c44c9a20 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:33.150436 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b2d8ec-2cd9-4237-8b71-1424c44c9a20 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:33.185443 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c6efdf-84e5-4e90-a0bb-4fc5ef1e6ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:33.186559 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6efdf-84e5-4e90-a0bb-4fc5ef1e6ca4 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:33.187040 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6efdf-84e5-4e90-a0bb-4fc5ef1e6ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:33.187561 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c6efdf-84e5-4e90-a0bb-4fc5ef1e6ca4 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:33.188557 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c6efdf-84e5-4e90-a0bb-4fc5ef1e6ca4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:33.190178 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c6efdf-84e5-4e90-a0bb-4fc5ef1e6ca4 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:33.204332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-21c6efdf-84e5-4e90-a0bb-4fc5ef1e6ca4 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 06 16:55:33.204332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:33.204332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 16:55:33.204332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 16:55:33.204332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 16:55:33.204332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 16:55:33.204332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 16:55:33.204332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:33.204332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 06 16:55:33.204332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:33.204332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 16:55:33.204332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:33.204332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:33.204332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:33.204332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:33.204332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:33.205379 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:33.205379 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:33.205379 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:33.205379 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:33.205379 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:33.205379 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:33.205379 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:33.205379 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 16:55:33.205379 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 16:55:33.205379 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 16:55:33.205379 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 16:55:33.205379 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:33.205379 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:33.205379 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:33.205379 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:33.205379 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:33.206299 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:33.206299 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:33.206299 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:33.206299 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:33.206299 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:33.206299 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:33.206299 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:33.206299 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:33.206299 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:33.206299 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:33.206299 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:33.206299 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 16:55:33.206299 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:33.206299 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 06 16:55:33.206299 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:33.209103 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 452/928] 10.222.0.40 () {56 vars in 1132 bytes} [Wed Jul 6 16:55:33 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:33.215661 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c0f86a-59db-4ca1-ab2d-25e5439ffb0d None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:33.264387 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:33.323250 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c0f86a-59db-4ca1-ab2d-25e5439ffb0d None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:33.359241 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c127b-e083-4665-a455-d9e54493f358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:33.360013 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c127b-e083-4665-a455-d9e54493f358 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:33.360296 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c127b-e083-4665-a455-d9e54493f358 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:33.360609 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c127b-e083-4665-a455-d9e54493f358 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:33.361326 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9c127b-e083-4665-a455-d9e54493f358 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:33.361636 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9c127b-e083-4665-a455-d9e54493f358 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:33.363043 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9c127b-e083-4665-a455-d9e54493f358 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:33.368511 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 477/929] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Jul 6 16:55:33 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:33.374040 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c6efdf-84e5-4e90-a0bb-4fc5ef1e6ca4 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:33.408105 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:33.485891 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c6efdf-84e5-4e90-a0bb-4fc5ef1e6ca4 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:33.520739 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9958cf6f-66bb-42a4-bd15-251f3aa1db80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:33.521673 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958cf6f-66bb-42a4-bd15-251f3aa1db80 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:33.521960 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958cf6f-66bb-42a4-bd15-251f3aa1db80 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:33.522768 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9958cf6f-66bb-42a4-bd15-251f3aa1db80 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:33.523481 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9958cf6f-66bb-42a4-bd15-251f3aa1db80 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:33.523815 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9958cf6f-66bb-42a4-bd15-251f3aa1db80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:33.525181 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9958cf6f-66bb-42a4-bd15-251f3aa1db80 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:33.530643 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 453/930] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Jul 6 16:55:33 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:33.536214 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c127b-e083-4665-a455-d9e54493f358 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:33.645634 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c127b-e083-4665-a455-d9e54493f358 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:33.658382 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:33.684462 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c1bd87-8298-4198-9e9f-a4f22a09d760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:33.685494 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1bd87-8298-4198-9e9f-a4f22a09d760 None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:33.685999 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1bd87-8298-4198-9e9f-a4f22a09d760 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:33.686483 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1bd87-8298-4198-9e9f-a4f22a09d760 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:33.687449 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c1bd87-8298-4198-9e9f-a4f22a09d760 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:33.689072 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c1bd87-8298-4198-9e9f-a4f22a09d760 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:33.766801 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 478/931] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 6 16:55:33 2022] POST /identity/v3/endpoints => generated 347 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 16:55:33.786173 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1507b-cc08-4a47-976a-8f73fc56320a None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:33.786924 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1507b-cc08-4a47-976a-8f73fc56320a None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:33.787466 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1507b-cc08-4a47-976a-8f73fc56320a None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:33.789629 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 454/932] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:33.795883 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542a8a0-e4ae-4a6c-b593-61fe6ecc7eff None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:33.796202 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542a8a0-e4ae-4a6c-b593-61fe6ecc7eff None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:33.796424 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542a8a0-e4ae-4a6c-b593-61fe6ecc7eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:33.816590 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-c542a8a0-e4ae-4a6c-b593-61fe6ecc7eff None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:33.817826 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c542a8a0-e4ae-4a6c-b593-61fe6ecc7eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:33.906060 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c021f-fd8f-4c71-a934-f62b01b34c4a None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:33.906436 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c021f-fd8f-4c71-a934-f62b01b34c4a None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:33.906685 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273c021f-fd8f-4c71-a934-f62b01b34c4a None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:33.907987 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 479/933] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:55:33.908117 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 455/934] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:33.913962 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80ca4f-b728-44a5-be6d-83674d4f96c7 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:33.914258 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80ca4f-b728-44a5-be6d-83674d4f96c7 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:33.914541 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80ca4f-b728-44a5-be6d-83674d4f96c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:33.932196 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-4e80ca4f-b728-44a5-be6d-83674d4f96c7 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:33.933391 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-4e80ca4f-b728-44a5-be6d-83674d4f96c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:33.933817 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273c021f-fd8f-4c71-a934-f62b01b34c4a None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:34.024568 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 480/935] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 16:55:34.047147 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e80ca4f-b728-44a5-be6d-83674d4f96c7 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:34.067750 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:34.076925 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273c021f-fd8f-4c71-a934-f62b01b34c4a None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:34.108159 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:34.126482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9091a70-1d1c-4c4f-841e-b06b4ac8d845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:34.127298 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9091a70-1d1c-4c4f-841e-b06b4ac8d845 None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:34.127689 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9091a70-1d1c-4c4f-841e-b06b4ac8d845 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:34.128092 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9091a70-1d1c-4c4f-841e-b06b4ac8d845 None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:34.129367 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9091a70-1d1c-4c4f-841e-b06b4ac8d845 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:34.131293 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9091a70-1d1c-4c4f-841e-b06b4ac8d845 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:34.133958 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: WARNING keystone.server.flask.common [None req-f9091a70-1d1c-4c4f-841e-b06b4ac8d845 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 06 16:55:34.156865 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e80ca4f-b728-44a5-be6d-83674d4f96c7 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:34.190829 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388aaa4e-bc31-4cff-95d3-574a67d52630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:34.191651 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388aaa4e-bc31-4cff-95d3-574a67d52630 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:34.191967 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388aaa4e-bc31-4cff-95d3-574a67d52630 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:34.192342 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388aaa4e-bc31-4cff-95d3-574a67d52630 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:34.193954 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.sql.core [None req-f9091a70-1d1c-4c4f-841e-b06b4ac8d845 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 06 16:55:34.193954 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 06 16:55:34.193954 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [parameters: {'id': '527d93dba5f54230a792b9b77cb118bf', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 06 16:55:34.193954 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80470) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 06 16:55:34.204171 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388aaa4e-bc31-4cff-95d3-574a67d52630 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:55:34.205058 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-f9091a70-1d1c-4c4f-841e-b06b4ac8d845 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 527d93dba5f54230a792b9b77cb118bf.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 527d93dba5f54230a792b9b77cb118bf. Jul 06 16:55:34.205058 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:34.205058 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 06 16:55:34.205058 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 06 16:55:34.205058 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 06 16:55:34.205058 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 06 16:55:34.205058 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 06 16:55:34.205058 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application result = self._query(query) Jul 06 16:55:34.205058 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 06 16:55:34.205058 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application conn.query(q) Jul 06 16:55:34.205058 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 06 16:55:34.205058 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 06 16:55:34.205058 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 06 16:55:34.205990 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application result.read() Jul 06 16:55:34.205990 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 06 16:55:34.205990 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 06 16:55:34.205990 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 06 16:55:34.205990 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 06 16:55:34.205990 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 06 16:55:34.205990 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 06 16:55:34.205990 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 06 16:55:34.205990 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 06 16:55:34.205990 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 06 16:55:34.205990 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:34.205990 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 06 16:55:34.205990 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:34.205990 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:34.205990 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 06 16:55:34.205990 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 06 16:55:34.205990 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 06 16:55:34.205990 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 06 16:55:34.206987 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 06 16:55:34.206987 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application next(self.gen) Jul 06 16:55:34.206987 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 06 16:55:34.206987 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application yield resource Jul 06 16:55:34.206987 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 06 16:55:34.206987 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application next(self.gen) Jul 06 16:55:34.206987 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 06 16:55:34.206987 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self.session.rollback() Jul 06 16:55:34.206987 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 06 16:55:34.206987 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self.force_reraise() Jul 06 16:55:34.206987 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 06 16:55:34.206987 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise self.value Jul 06 16:55:34.206987 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 06 16:55:34.206987 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 06 16:55:34.206987 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 06 16:55:34.206987 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application session.commit() Jul 06 16:55:34.206987 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 06 16:55:34.207937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 06 16:55:34.207937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 06 16:55:34.207937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._prepare_impl() Jul 06 16:55:34.207937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 06 16:55:34.207937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self.session.flush() Jul 06 16:55:34.207937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 06 16:55:34.207937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._flush(objects) Jul 06 16:55:34.207937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 06 16:55:34.207937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 06 16:55:34.207937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 06 16:55:34.207937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application compat.raise_( Jul 06 16:55:34.207937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 06 16:55:34.207937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception Jul 06 16:55:34.207937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 06 16:55:34.207937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application flush_context.execute() Jul 06 16:55:34.207937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 06 16:55:34.207937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rec.execute(self) Jul 06 16:55:34.208900 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 06 16:55:34.208900 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 06 16:55:34.208900 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 06 16:55:34.208900 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 06 16:55:34.208900 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 06 16:55:34.208900 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 06 16:55:34.208900 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 06 16:55:34.208900 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 06 16:55:34.208900 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 06 16:55:34.208900 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 06 16:55:34.208900 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 06 16:55:34.208900 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 06 16:55:34.208900 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 06 16:55:34.208900 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 06 16:55:34.208900 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 06 16:55:34.211185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 06 16:55:34.211185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 06 16:55:34.211185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception Jul 06 16:55:34.211185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 06 16:55:34.211185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 06 16:55:34.211185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 06 16:55:34.211185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 06 16:55:34.211185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 06 16:55:34.211185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application result = self._query(query) Jul 06 16:55:34.211185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 06 16:55:34.211185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application conn.query(q) Jul 06 16:55:34.211185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 06 16:55:34.211185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 06 16:55:34.211185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 06 16:55:34.211185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application result.read() Jul 06 16:55:34.211185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 06 16:55:34.211185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 06 16:55:34.213107 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 06 16:55:34.213107 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 06 16:55:34.213107 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 06 16:55:34.213107 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 06 16:55:34.213107 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 06 16:55:34.213107 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 06 16:55:34.213107 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 06 16:55:34.213107 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 06 16:55:34.213107 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [parameters: {'id': '527d93dba5f54230a792b9b77cb118bf', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 06 16:55:34.213107 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 06 16:55:34.213107 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:34.213107 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 16:55:34.213107 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:34.213107 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:34.213107 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 06 16:55:34.214799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 06 16:55:34.214799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 06 16:55:34.214799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 06 16:55:34.214799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 06 16:55:34.214799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:34.214799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 16:55:34.214799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:34.214799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:34.214799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:34.214799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:34.214799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:34.214799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:34.214799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:34.214799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:34.214799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:34.214799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:34.214799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:34.216780 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:34.216780 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 06 16:55:34.216780 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 06 16:55:34.216780 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:34.216780 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:34.216780 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 06 16:55:34.216780 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 06 16:55:34.216780 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 527d93dba5f54230a792b9b77cb118bf. Jul 06 16:55:34.216780 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:34.223916 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-388aaa4e-bc31-4cff-95d3-574a67d52630 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 06 16:55:34.223916 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:34.223916 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:34.223916 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:34.223916 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:34.223916 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:34.223916 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:34.223916 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:34.223916 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:34.223916 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:34.223916 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:34.223916 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:34.223916 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:55:34.223916 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:55:34.223916 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:55:34.223916 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:55:34.226387 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:55:34.226387 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:55:34.226387 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:55:34.226387 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:55:34.226387 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:34.226387 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:34.226387 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:34.226387 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:34.226387 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:34.226387 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:34.226387 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:34.226387 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:34.226387 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:34.226387 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:34.226387 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:34.226387 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:34.229439 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:34.229439 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:34.229439 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:34.229439 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:34.229439 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:55:34.229439 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:55:34.229439 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:55:34.229439 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:55:34.229439 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:55:34.229439 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:55:34.229439 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 06 16:55:34.229439 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:34.229439 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 481/936] 10.222.0.40 () {56 vars in 1120 bytes} [Wed Jul 6 16:55:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:55:34.234599 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 456/937] 10.222.0.40 () {60 vars in 1140 bytes} [Wed Jul 6 16:55:33 2022] POST /identity/v3/projects => generated 262 bytes in 287 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:55:34.234766 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388aaa4e-bc31-4cff-95d3-574a67d52630 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:34.234936 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9091a70-1d1c-4c4f-841e-b06b4ac8d845 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:34.322901 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:34.341936 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388aaa4e-bc31-4cff-95d3-574a67d52630 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:34.378867 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df26f1d7-1373-409c-a9db-696bdc1b4524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:34.379544 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26f1d7-1373-409c-a9db-696bdc1b4524 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:34.379860 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26f1d7-1373-409c-a9db-696bdc1b4524 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:34.380140 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26f1d7-1373-409c-a9db-696bdc1b4524 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:34.380896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df26f1d7-1373-409c-a9db-696bdc1b4524 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:34.381205 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df26f1d7-1373-409c-a9db-696bdc1b4524 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:34.382429 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9091a70-1d1c-4c4f-841e-b06b4ac8d845 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:34.382638 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df26f1d7-1373-409c-a9db-696bdc1b4524 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:34.395883 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 482/938] 10.222.0.40 () {56 vars in 1115 bytes} [Wed Jul 6 16:55:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:34.398535 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df26f1d7-1373-409c-a9db-696bdc1b4524 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:34.426432 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cd1ce0-318d-41eb-9f03-6138506c380f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:34.427374 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd1ce0-318d-41eb-9f03-6138506c380f None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:34.427782 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd1ce0-318d-41eb-9f03-6138506c380f None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:34.428164 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd1ce0-318d-41eb-9f03-6138506c380f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:34.435532 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cd1ce0-318d-41eb-9f03-6138506c380f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:34.437529 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cd1ce0-318d-41eb-9f03-6138506c380f None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:34.452116 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-80cd1ce0-318d-41eb-9f03-6138506c380f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:34.452116 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:34.452116 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:34.452116 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:34.452116 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:34.452116 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:34.452116 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:34.452116 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:34.452116 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:34.452116 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:34.452116 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:34.452116 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:34.452116 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 16:55:34.452116 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 16:55:34.452116 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 16:55:34.453241 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 16:55:34.453241 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:34.453241 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:34.453241 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:34.453241 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:34.453241 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:34.453241 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:34.453241 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:34.453241 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:34.453241 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:34.453241 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:34.453241 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:34.453241 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:34.453241 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:34.453241 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:34.453241 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:34.453241 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:34.454332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 16:55:34.454332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 16:55:34.454332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 16:55:34.454332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 16:55:34.454332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 16:55:34.454332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 16:55:34.454332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:34.454332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:34.456133 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 457/939] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:55:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:34.464873 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cd1ce0-318d-41eb-9f03-6138506c380f None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:34.488285 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:34.523256 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df26f1d7-1373-409c-a9db-696bdc1b4524 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:34.563524 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e6764-800f-49c0-8bcd-de104fa5dc49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:34.564363 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e6764-800f-49c0-8bcd-de104fa5dc49 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:34.564649 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e6764-800f-49c0-8bcd-de104fa5dc49 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:34.564929 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671e6764-800f-49c0-8bcd-de104fa5dc49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:34.579430 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671e6764-800f-49c0-8bcd-de104fa5dc49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:34.580866 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671e6764-800f-49c0-8bcd-de104fa5dc49 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:34.583980 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:34.600155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-671e6764-800f-49c0-8bcd-de104fa5dc49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:34.600155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:34.600155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:34.600155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:34.600155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:34.600155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:34.600155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:34.600155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:34.600155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:34.600155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:34.600155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:34.600155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:34.600155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:55:34.600155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:55:34.600155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:55:34.601131 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:55:34.601131 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:34.601131 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:34.601131 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:34.601131 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:34.601131 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:34.601131 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:34.601131 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:34.601131 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:34.601131 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:34.601131 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:34.601131 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:34.601131 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:34.601131 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:34.601131 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:34.601131 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:34.601131 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:34.602651 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:55:34.602651 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:55:34.602651 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:34.602651 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:34.602651 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 483/940] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:34.604107 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cd1ce0-318d-41eb-9f03-6138506c380f None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:34.607540 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e6764-800f-49c0-8bcd-de104fa5dc49 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:34.648569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f2d2e-a48f-448d-918d-bc942693841d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:34.649242 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f2d2e-a48f-448d-918d-bc942693841d None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:34.649529 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f2d2e-a48f-448d-918d-bc942693841d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:34.649809 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680f2d2e-a48f-448d-918d-bc942693841d None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:34.650569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680f2d2e-a48f-448d-918d-bc942693841d None admin] RBAC: Adding query filter params (name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:34.650877 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680f2d2e-a48f-448d-918d-bc942693841d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:34.652408 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680f2d2e-a48f-448d-918d-bc942693841d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:34.679758 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 458/941] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Jul 6 16:55:34 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:34.751753 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671e6764-800f-49c0-8bcd-de104fa5dc49 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:34.791791 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ffd173-188c-4593-a82d-5ba71cae8efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:34.792489 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ffd173-188c-4593-a82d-5ba71cae8efa None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:34.792777 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ffd173-188c-4593-a82d-5ba71cae8efa None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:34.793058 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ffd173-188c-4593-a82d-5ba71cae8efa None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:34.793804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ffd173-188c-4593-a82d-5ba71cae8efa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:34.794175 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ffd173-188c-4593-a82d-5ba71cae8efa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:34.795572 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ffd173-188c-4593-a82d-5ba71cae8efa None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:34.809906 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 484/942] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:55:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:34.815336 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f2d2e-a48f-448d-918d-bc942693841d None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:34.896537 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:34.929775 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680f2d2e-a48f-448d-918d-bc942693841d None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:34.945847 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78524e20-4592-4550-bb23-d0a5988ae16f None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:34.946140 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78524e20-4592-4550-bb23-d0a5988ae16f None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:34.946419 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78524e20-4592-4550-bb23-d0a5988ae16f None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:34.948412 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 485/943] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:34.954283 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2387be-8aa6-4207-b281-8210b141c665 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:34.954283 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2387be-8aa6-4207-b281-8210b141c665 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:34.954490 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2387be-8aa6-4207-b281-8210b141c665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:34.975884 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-db2387be-8aa6-4207-b281-8210b141c665 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:34.976858 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3331849-be04-4777-a1ee-f4b697d61fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:34.977087 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-db2387be-8aa6-4207-b281-8210b141c665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:34.977684 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3331849-be04-4777-a1ee-f4b697d61fbf None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:34.977968 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3331849-be04-4777-a1ee-f4b697d61fbf None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:34.978240 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3331849-be04-4777-a1ee-f4b697d61fbf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:34.987055 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3331849-be04-4777-a1ee-f4b697d61fbf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:34.988421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3331849-be04-4777-a1ee-f4b697d61fbf None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:35.003545 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-a3331849-be04-4777-a1ee-f4b697d61fbf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 06 16:55:35.003545 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:35.003545 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:35.003545 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:35.003545 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:35.003545 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:35.003545 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:35.003545 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:35.003545 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:35.003545 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:35.003545 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:35.003545 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:35.003545 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 16:55:35.003545 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 16:55:35.003545 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 16:55:35.003545 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 16:55:35.004766 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:35.004766 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:35.004766 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 16:55:35.004766 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:35.004766 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 16:55:35.004766 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:35.004766 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:35.004766 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:35.004766 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:35.004766 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:35.004766 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:35.004766 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:35.004766 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:35.004766 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:35.004766 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:35.004766 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:35.004766 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:35.004766 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:35.006079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:35.006079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:35.006079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 16:55:35.006079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 16:55:35.006079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 16:55:35.006079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 16:55:35.006079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 16:55:35.006079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 16:55:35.006079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 06 16:55:35.006079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:35.006079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 459/944] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Jul 6 16:55:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:35.010574 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3331849-be04-4777-a1ee-f4b697d61fbf None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:35.044986 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:35.058680 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 486/945] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:34 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 16:55:35.086633 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2387be-8aa6-4207-b281-8210b141c665 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:35.140670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3331849-be04-4777-a1ee-f4b697d61fbf None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:35.176024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:35.181933 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5dd44e-6b4c-4653-aa73-2a7876039cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:35.187068 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5dd44e-6b4c-4653-aa73-2a7876039cde None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:35.187068 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5dd44e-6b4c-4653-aa73-2a7876039cde None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:35.187068 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5dd44e-6b4c-4653-aa73-2a7876039cde None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:35.187068 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5dd44e-6b4c-4653-aa73-2a7876039cde None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:35.187068 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5dd44e-6b4c-4653-aa73-2a7876039cde None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:35.187068 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5dd44e-6b4c-4653-aa73-2a7876039cde None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:35.197412 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2387be-8aa6-4207-b281-8210b141c665 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:35.208007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 460/946] 10.222.0.40 () {56 vars in 1149 bytes} [Wed Jul 6 16:55:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:35.211147 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5dd44e-6b4c-4653-aa73-2a7876039cde None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:35.239929 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecca0a7-9d9c-4db1-9085-98ea0c8ed26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:35.244343 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecca0a7-9d9c-4db1-9085-98ea0c8ed26d None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:35.244343 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecca0a7-9d9c-4db1-9085-98ea0c8ed26d None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:35.244343 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecca0a7-9d9c-4db1-9085-98ea0c8ed26d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:35.244343 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecca0a7-9d9c-4db1-9085-98ea0c8ed26d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:35.245671 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecca0a7-9d9c-4db1-9085-98ea0c8ed26d None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:35.261168 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-cecca0a7-9d9c-4db1-9085-98ea0c8ed26d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 16:55:35.261168 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:35.261168 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 16:55:35.261168 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 16:55:35.261168 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 16:55:35.261168 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 16:55:35.261168 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 16:55:35.261168 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:35.261168 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 16:55:35.261168 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:35.261168 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 16:55:35.261168 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:35.261168 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:35.261168 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:35.261168 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:35.261168 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:35.261168 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:35.262453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:35.262453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:35.262453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:35.262453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:35.262453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:35.262453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:35.262453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 16:55:35.262453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 16:55:35.262453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 16:55:35.262453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 16:55:35.262453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:35.262453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:35.262453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:35.262453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:35.262453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:35.262453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:35.263491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:35.263491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:35.263491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:35.263491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:35.263491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:35.263491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:35.263491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:35.263491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:35.263491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:35.263491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:35.263491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 16:55:35.263491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:35.263491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 16:55:35.263491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:35.271685 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 487/947] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Jul 6 16:55:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:55:35.274102 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecca0a7-9d9c-4db1-9085-98ea0c8ed26d None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:35.330117 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5dd44e-6b4c-4653-aa73-2a7876039cde None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:35.341945 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:35.374781 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1267d0e-995e-4b9a-98a8-edfff8475fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:35.375594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1267d0e-995e-4b9a-98a8-edfff8475fbb None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:35.375920 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1267d0e-995e-4b9a-98a8-edfff8475fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:35.376247 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1267d0e-995e-4b9a-98a8-edfff8475fbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:35.388759 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1267d0e-995e-4b9a-98a8-edfff8475fbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:35.390373 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1267d0e-995e-4b9a-98a8-edfff8475fbb None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:35.406346 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecca0a7-9d9c-4db1-9085-98ea0c8ed26d None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:35.410312 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-c1267d0e-995e-4b9a-98a8-edfff8475fbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:35.410312 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:35.410312 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:35.410312 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:35.410312 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:35.410312 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:35.410312 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:35.410312 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:35.410312 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:35.410312 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:35.410312 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:35.410312 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:35.410312 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:55:35.410312 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:55:35.410312 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:55:35.411170 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:55:35.411170 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:35.411170 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:35.411170 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:35.411170 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:35.411170 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:35.411170 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:35.411170 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:35.411170 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:35.411170 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:35.411170 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:35.411170 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:35.411170 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:35.411170 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:35.411170 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:35.411170 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:35.411170 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:35.412105 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:55:35.412105 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:55:35.412105 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:35.412105 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:35.416056 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 461/948] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:35.417614 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1267d0e-995e-4b9a-98a8-edfff8475fbb None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:35.450661 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39d7ee-4c75-4a48-a8a2-a38d0686de76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:35.451878 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39d7ee-4c75-4a48-a8a2-a38d0686de76 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:35.452645 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39d7ee-4c75-4a48-a8a2-a38d0686de76 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:35.453386 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39d7ee-4c75-4a48-a8a2-a38d0686de76 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:35.454319 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f39d7ee-4c75-4a48-a8a2-a38d0686de76 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:35.454840 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f39d7ee-4c75-4a48-a8a2-a38d0686de76 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:35.457414 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f39d7ee-4c75-4a48-a8a2-a38d0686de76 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:35.468859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 488/949] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Jul 6 16:55:35 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:35.470662 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:35.473028 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39d7ee-4c75-4a48-a8a2-a38d0686de76 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:35.535868 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1267d0e-995e-4b9a-98a8-edfff8475fbb None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:35.574278 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b27933-a9e0-4a07-a797-5337dd655266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:35.575082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b27933-a9e0-4a07-a797-5337dd655266 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:35.575373 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b27933-a9e0-4a07-a797-5337dd655266 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:35.575648 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b27933-a9e0-4a07-a797-5337dd655266 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:35.576482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b27933-a9e0-4a07-a797-5337dd655266 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:35.576793 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b27933-a9e0-4a07-a797-5337dd655266 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:35.578335 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b27933-a9e0-4a07-a797-5337dd655266 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:35.596218 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 462/950] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:55:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:35.597011 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b27933-a9e0-4a07-a797-5337dd655266 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:35.603389 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39d7ee-4c75-4a48-a8a2-a38d0686de76 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:35.620345 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:35.645742 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed91318a-0504-4f44-a0a8-99674a7a9a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:35.646949 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91318a-0504-4f44-a0a8-99674a7a9a20 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:35.647990 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91318a-0504-4f44-a0a8-99674a7a9a20 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:35.648337 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91318a-0504-4f44-a0a8-99674a7a9a20 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:35.649115 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed91318a-0504-4f44-a0a8-99674a7a9a20 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:35.649492 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed91318a-0504-4f44-a0a8-99674a7a9a20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:35.651601 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed91318a-0504-4f44-a0a8-99674a7a9a20 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:35.659794 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 489/951] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Jul 6 16:55:35 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:35.665637 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed91318a-0504-4f44-a0a8-99674a7a9a20 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:35.716258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b27933-a9e0-4a07-a797-5337dd655266 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:35.718185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:35.769332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a2201-0aa5-48dc-b88e-49dc4f17a674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:35.770142 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a2201-0aa5-48dc-b88e-49dc4f17a674 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:35.770414 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a2201-0aa5-48dc-b88e-49dc4f17a674 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:35.770688 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a2201-0aa5-48dc-b88e-49dc4f17a674 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:35.780065 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782a2201-0aa5-48dc-b88e-49dc4f17a674 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:35.781933 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782a2201-0aa5-48dc-b88e-49dc4f17a674 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:35.798748 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed91318a-0504-4f44-a0a8-99674a7a9a20 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:35.799408 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-782a2201-0aa5-48dc-b88e-49dc4f17a674 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:35.799408 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:35.799408 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:35.799408 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:35.799408 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:35.799408 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:35.799408 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:35.799408 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:35.799408 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:35.799408 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:35.799408 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:35.799408 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:35.799408 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 16:55:35.799408 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 16:55:35.799408 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 16:55:35.800384 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 16:55:35.800384 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:35.800384 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:35.800384 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:35.800384 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:35.800384 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:35.800384 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:35.800384 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:35.800384 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:35.800384 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:35.800384 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:35.800384 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:35.800384 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:35.800384 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:35.800384 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:35.800384 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:35.800384 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:35.801372 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 16:55:35.801372 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 16:55:35.801372 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 16:55:35.801372 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 16:55:35.801372 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 16:55:35.801372 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 16:55:35.801372 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:35.801372 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:35.801372 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 463/952] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:55:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:35.807959 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a2201-0aa5-48dc-b88e-49dc4f17a674 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:35.841963 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c33a7-2cd3-4d4d-97d2-b691742ca960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:35.842960 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c33a7-2cd3-4d4d-97d2-b691742ca960 None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:35.843267 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c33a7-2cd3-4d4d-97d2-b691742ca960 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:35.843549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c33a7-2cd3-4d4d-97d2-b691742ca960 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:35.844432 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510c33a7-2cd3-4d4d-97d2-b691742ca960 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:35.845828 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510c33a7-2cd3-4d4d-97d2-b691742ca960 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:35.903227 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 490/953] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 6 16:55:35 2022] POST /identity/v3/endpoints => generated 349 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 16:55:35.904579 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee67cc-6b30-4da5-a554-520972614a0e None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:35.904869 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee67cc-6b30-4da5-a554-520972614a0e None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:35.905150 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ee67cc-6b30-4da5-a554-520972614a0e None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:35.906619 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 491/954] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:35.912142 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d7dbc-77b2-468c-803a-6961d9866940 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:35.912452 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d7dbc-77b2-468c-803a-6961d9866940 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:35.912739 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102d7dbc-77b2-468c-803a-6961d9866940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:35.931604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-102d7dbc-77b2-468c-803a-6961d9866940 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:35.932932 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-102d7dbc-77b2-468c-803a-6961d9866940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:35.938327 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a2201-0aa5-48dc-b88e-49dc4f17a674 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:35.968596 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:35.994333 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52fdc7-6c9c-413b-a45a-9faf58b7a13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:35.995990 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52fdc7-6c9c-413b-a45a-9faf58b7a13f None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:35.996761 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52fdc7-6c9c-413b-a45a-9faf58b7a13f None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:35.997254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe52fdc7-6c9c-413b-a45a-9faf58b7a13f None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:35.998848 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe52fdc7-6c9c-413b-a45a-9faf58b7a13f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:35.999720 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe52fdc7-6c9c-413b-a45a-9faf58b7a13f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:36.002112 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe52fdc7-6c9c-413b-a45a-9faf58b7a13f None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:36.016298 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 492/955] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 16:55:36.017809 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 464/956] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 6 16:55:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:36.023970 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102d7dbc-77b2-468c-803a-6961d9866940 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:36.035220 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52fdc7-6c9c-413b-a45a-9faf58b7a13f None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:36.066325 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:36.153067 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102d7dbc-77b2-468c-803a-6961d9866940 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:36.172661 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe52fdc7-6c9c-413b-a45a-9faf58b7a13f None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:36.198508 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171cd61f-2ab4-42a7-8333-29004222b636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:36.199434 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171cd61f-2ab4-42a7-8333-29004222b636 None admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:36.199719 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171cd61f-2ab4-42a7-8333-29004222b636 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:36.200031 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171cd61f-2ab4-42a7-8333-29004222b636 None admin] PATH_INFO: `/v3/projects/7938b0346d4240ab89774327d09a89ae/users/1905e6f8c3e24e3cbfefce2827baa370/roles/4d3a77c1547c43aba11b60377d04d537` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:36.202584 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171cd61f-2ab4-42a7-8333-29004222b636 None admin] RBAC: Authorizing `identity:create_grant(project_id=7938b0346d4240ab89774327d09a89ae, user_id=1905e6f8c3e24e3cbfefce2827baa370, role_id=4d3a77c1547c43aba11b60377d04d537)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:36.204566 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171cd61f-2ab4-42a7-8333-29004222b636 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:36.215785 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aa748e-5f76-42c5-947f-6e7fc12dbb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:36.216650 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa748e-5f76-42c5-947f-6e7fc12dbb1a None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:36.216940 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa748e-5f76-42c5-947f-6e7fc12dbb1a None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:36.217265 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82aa748e-5f76-42c5-947f-6e7fc12dbb1a None admin] PATH_INFO: `/v3/services/compute` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:36.217930 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aa748e-5f76-42c5-947f-6e7fc12dbb1a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:36.219330 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82aa748e-5f76-42c5-947f-6e7fc12dbb1a None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:36.232007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-82aa748e-5f76-42c5-947f-6e7fc12dbb1a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 06 16:55:36.232007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:36.232007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 16:55:36.232007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 16:55:36.232007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 16:55:36.232007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 16:55:36.232007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 16:55:36.232007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:36.232007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 06 16:55:36.232007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:36.232007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 16:55:36.232007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:36.232007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:36.232007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:36.232007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:36.232007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:36.232007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:36.233169 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:36.233169 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:36.233169 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:36.233169 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:36.233169 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:36.233169 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:36.233169 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 16:55:36.233169 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 16:55:36.233169 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 16:55:36.233169 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 16:55:36.233169 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:36.233169 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:36.233169 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:36.233169 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:36.233169 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:36.233169 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:36.234639 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:36.234639 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:36.234639 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:36.234639 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:36.234639 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:36.234639 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:36.234639 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:36.234639 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:36.234639 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:36.234639 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:36.234639 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 16:55:36.234639 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:36.234639 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 06 16:55:36.234639 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:36.235815 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 465/957] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:55:36 2022] GET /identity/v3/services/compute => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:55:36.239198 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aa748e-5f76-42c5-947f-6e7fc12dbb1a None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:36.289402 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 493/958] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 6 16:55:36 2022] PUT /identity/v3/projects/7938b0346d4240ab89774327d09a89ae/users/1905e6f8c3e24e3cbfefce2827baa370/roles/4d3a77c1547c43aba11b60377d04d537 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 16:55:36.339941 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:36.361720 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82aa748e-5f76-42c5-947f-6e7fc12dbb1a None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:36.401937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc30408-8751-46e0-9f94-84cb603759b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:36.402704 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc30408-8751-46e0-9f94-84cb603759b1 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:36.402998 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc30408-8751-46e0-9f94-84cb603759b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:36.403319 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc30408-8751-46e0-9f94-84cb603759b1 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:36.404085 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc30408-8751-46e0-9f94-84cb603759b1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:36.404397 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc30408-8751-46e0-9f94-84cb603759b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:36.405872 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc30408-8751-46e0-9f94-84cb603759b1 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:36.412154 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 466/959] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Jul 6 16:55:36 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:36.418630 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171cd61f-2ab4-42a7-8333-29004222b636 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:36.448166 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:36.528326 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171cd61f-2ab4-42a7-8333-29004222b636 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:36.568039 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b751941-389f-4e76-a2de-4e1868d6fc34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:36.569662 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b751941-389f-4e76-a2de-4e1868d6fc34 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:36.570160 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b751941-389f-4e76-a2de-4e1868d6fc34 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:36.570663 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b751941-389f-4e76-a2de-4e1868d6fc34 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:36.571602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b751941-389f-4e76-a2de-4e1868d6fc34 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:36.572145 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b751941-389f-4e76-a2de-4e1868d6fc34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:36.573731 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b751941-389f-4e76-a2de-4e1868d6fc34 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:36.579328 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 494/960] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Jul 6 16:55:36 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:36.856860 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f775f2-ba04-48b5-8c2b-0a3f3771e7d2 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:36.857201 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f775f2-ba04-48b5-8c2b-0a3f3771e7d2 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:36.857324 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f775f2-ba04-48b5-8c2b-0a3f3771e7d2 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:36.858909 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 467/961] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:36.864136 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9dd941-c0a0-404b-916e-eeb43943b6ae None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:36.864431 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9dd941-c0a0-404b-916e-eeb43943b6ae None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:36.864711 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9dd941-c0a0-404b-916e-eeb43943b6ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:36.881976 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-ae9dd941-c0a0-404b-916e-eeb43943b6ae None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:36.883145 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-ae9dd941-c0a0-404b-916e-eeb43943b6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:36.957050 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 495/962] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:36 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:55:36.981054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f775f2-ba04-48b5-8c2b-0a3f3771e7d2 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:37.086341 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f775f2-ba04-48b5-8c2b-0a3f3771e7d2 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:37.124647 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21424646-a6e4-4a55-8097-01f752693f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:37.125549 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21424646-a6e4-4a55-8097-01f752693f78 None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:37.125999 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21424646-a6e4-4a55-8097-01f752693f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:37.126426 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21424646-a6e4-4a55-8097-01f752693f78 None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:37.127281 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21424646-a6e4-4a55-8097-01f752693f78 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:37.128921 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21424646-a6e4-4a55-8097-01f752693f78 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:37.185262 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.sql.core [None req-21424646-a6e4-4a55-8097-01f752693f78 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 06 16:55:37.185262 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 06 16:55:37.185262 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [parameters: {'user_id': 'e49275a28d9d443e9e2eead71df24f02', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 06 16:55:37.185262 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80470) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 06 16:55:37.193313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-21424646-a6e4-4a55-8097-01f752693f78 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 06 16:55:37.193313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:37.193313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 06 16:55:37.193313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 06 16:55:37.193313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 06 16:55:37.193313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 06 16:55:37.193313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 06 16:55:37.193313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application result = self._query(query) Jul 06 16:55:37.193313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 06 16:55:37.193313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application conn.query(q) Jul 06 16:55:37.193313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 06 16:55:37.193313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 06 16:55:37.193313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 06 16:55:37.193313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application result.read() Jul 06 16:55:37.193313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 06 16:55:37.194427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 06 16:55:37.194427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 06 16:55:37.194427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 06 16:55:37.194427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 06 16:55:37.194427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 06 16:55:37.194427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 06 16:55:37.194427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 06 16:55:37.194427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 06 16:55:37.194427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:37.194427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 06 16:55:37.194427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:37.194427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:37.194427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 06 16:55:37.194427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 06 16:55:37.194427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 06 16:55:37.194427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 06 16:55:37.194427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 06 16:55:37.194427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application next(self.gen) Jul 06 16:55:37.195600 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 06 16:55:37.195600 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application yield resource Jul 06 16:55:37.195600 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 06 16:55:37.195600 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application next(self.gen) Jul 06 16:55:37.195600 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 06 16:55:37.195600 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self.session.rollback() Jul 06 16:55:37.195600 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 06 16:55:37.195600 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self.force_reraise() Jul 06 16:55:37.195600 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 06 16:55:37.195600 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise self.value Jul 06 16:55:37.195600 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 06 16:55:37.195600 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 06 16:55:37.195600 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 06 16:55:37.195600 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application session.commit() Jul 06 16:55:37.195600 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 06 16:55:37.195600 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 06 16:55:37.195600 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 06 16:55:37.196721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._prepare_impl() Jul 06 16:55:37.196721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 06 16:55:37.196721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self.session.flush() Jul 06 16:55:37.196721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 06 16:55:37.196721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._flush(objects) Jul 06 16:55:37.196721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 06 16:55:37.196721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 06 16:55:37.196721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 06 16:55:37.196721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application compat.raise_( Jul 06 16:55:37.196721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 06 16:55:37.196721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception Jul 06 16:55:37.196721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 06 16:55:37.196721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application flush_context.execute() Jul 06 16:55:37.196721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 06 16:55:37.196721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rec.execute(self) Jul 06 16:55:37.196721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 06 16:55:37.196721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 06 16:55:37.197834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 06 16:55:37.197834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 06 16:55:37.197834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 06 16:55:37.197834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 06 16:55:37.197834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 06 16:55:37.197834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 06 16:55:37.197834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 06 16:55:37.197834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 06 16:55:37.197834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 06 16:55:37.197834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 06 16:55:37.197834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 06 16:55:37.197834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 06 16:55:37.197834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 06 16:55:37.197834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 06 16:55:37.197834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 06 16:55:37.197834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception Jul 06 16:55:37.198802 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 06 16:55:37.198802 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 06 16:55:37.198802 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 06 16:55:37.198802 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 06 16:55:37.198802 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 06 16:55:37.198802 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application result = self._query(query) Jul 06 16:55:37.198802 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 06 16:55:37.198802 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application conn.query(q) Jul 06 16:55:37.198802 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 06 16:55:37.198802 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 06 16:55:37.198802 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 06 16:55:37.198802 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application result.read() Jul 06 16:55:37.198802 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 06 16:55:37.198802 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 06 16:55:37.198802 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 06 16:55:37.198802 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 06 16:55:37.199774 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 06 16:55:37.199774 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 06 16:55:37.199774 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 06 16:55:37.199774 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 06 16:55:37.199774 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 06 16:55:37.199774 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 06 16:55:37.199774 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [parameters: {'user_id': 'e49275a28d9d443e9e2eead71df24f02', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 06 16:55:37.199774 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 06 16:55:37.199774 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:37.199774 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 16:55:37.199774 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:37.199774 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:37.199774 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:37.199774 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:37.199774 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:37.200749 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:37.200749 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:37.200749 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:37.200749 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:37.200749 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:37.200749 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:37.200749 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:37.200749 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 06 16:55:37.200749 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 06 16:55:37.200749 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:37.200749 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:37.200749 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 16:55:37.200749 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:37.200749 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 16:55:37.200749 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:37.200749 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 06 16:55:37.200749 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 06 16:55:37.201838 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 06 16:55:37.201838 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 06 16:55:37.201838 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 06 16:55:37.201838 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 06 16:55:37.201838 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 06 16:55:37.201838 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:37.203925 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 468/963] 10.222.0.40 () {60 vars in 1131 bytes} [Wed Jul 6 16:55:36 2022] POST /identity/v3/users => generated 160 bytes in 223 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:37.209481 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9dd941-c0a0-404b-916e-eeb43943b6ae None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:37.229185 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:37.314552 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9dd941-c0a0-404b-916e-eeb43943b6ae None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:37.350388 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada4b623-389a-4fc1-8f62-d6961a00a9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:37.351193 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada4b623-389a-4fc1-8f62-d6961a00a9c7 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:37.351481 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada4b623-389a-4fc1-8f62-d6961a00a9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:37.351785 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada4b623-389a-4fc1-8f62-d6961a00a9c7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:37.359169 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada4b623-389a-4fc1-8f62-d6961a00a9c7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:37.360646 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada4b623-389a-4fc1-8f62-d6961a00a9c7 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:37.376831 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-ada4b623-389a-4fc1-8f62-d6961a00a9c7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 06 16:55:37.376831 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:37.376831 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:37.376831 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:37.376831 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:37.376831 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:37.376831 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:37.376831 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:37.376831 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:37.376831 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:37.376831 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:37.376831 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:37.376831 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 16:55:37.376831 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 16:55:37.376831 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 16:55:37.376831 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 16:55:37.377775 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:37.377775 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:37.377775 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 16:55:37.377775 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:37.377775 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 16:55:37.377775 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:37.377775 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:37.377775 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:37.377775 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:37.377775 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:37.377775 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:37.377775 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:37.377775 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:37.377775 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:37.377775 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:37.377775 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:37.377775 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:37.377775 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:37.378801 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:37.378801 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:37.378801 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 16:55:37.378801 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 16:55:37.378801 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 16:55:37.378801 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 16:55:37.378801 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 16:55:37.378801 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 16:55:37.378801 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 06 16:55:37.378801 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:37.378801 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 496/964] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:55:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:37.384585 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21424646-a6e4-4a55-8097-01f752693f78 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:37.396927 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:37.492057 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21424646-a6e4-4a55-8097-01f752693f78 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:37.532021 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5a220a-c352-4e27-a14d-0392c479c50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:37.532690 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a220a-c352-4e27-a14d-0392c479c50d None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:37.532968 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a220a-c352-4e27-a14d-0392c479c50d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:37.533241 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5a220a-c352-4e27-a14d-0392c479c50d None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:37.534209 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5a220a-c352-4e27-a14d-0392c479c50d None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:37.534518 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5a220a-c352-4e27-a14d-0392c479c50d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:37.536177 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5a220a-c352-4e27-a14d-0392c479c50d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:37.574088 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 469/965] 10.222.0.40 () {56 vars in 1101 bytes} [Wed Jul 6 16:55:37 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:38.142763 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc96f0-15dc-4bef-8d83-be9a898cde8d None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:38.142957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc96f0-15dc-4bef-8d83-be9a898cde8d None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:38.143247 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dc96f0-15dc-4bef-8d83-be9a898cde8d None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:38.144904 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 497/966] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:38.150991 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037ebed-854e-4328-9b9b-209cb41b2952 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:38.151279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037ebed-854e-4328-9b9b-209cb41b2952 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:38.151562 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1037ebed-854e-4328-9b9b-209cb41b2952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:38.168740 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-1037ebed-854e-4328-9b9b-209cb41b2952 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:38.170224 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-1037ebed-854e-4328-9b9b-209cb41b2952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:38.230769 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:38.238739 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 470/967] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:55:38.260897 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc96f0-15dc-4bef-8d83-be9a898cde8d None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:38.341655 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:38.364735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dc96f0-15dc-4bef-8d83-be9a898cde8d None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:38.371390 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208bdb8a-842c-4daf-9b8a-a5923efffd74 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:38.371690 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208bdb8a-842c-4daf-9b8a-a5923efffd74 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:38.372211 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208bdb8a-842c-4daf-9b8a-a5923efffd74 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:38.373518 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 471/968] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 16:55:38.378427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d299c38-0b03-4839-9c3e-f41804667d1f None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:38.378742 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d299c38-0b03-4839-9c3e-f41804667d1f None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:38.379028 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d299c38-0b03-4839-9c3e-f41804667d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:38.396642 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-3d299c38-0b03-4839-9c3e-f41804667d1f None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:38.398287 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-3d299c38-0b03-4839-9c3e-f41804667d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:38.401832 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b03cbb-e5f3-436f-b793-9575291b8139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:38.402646 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b03cbb-e5f3-436f-b793-9575291b8139 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:38.402939 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b03cbb-e5f3-436f-b793-9575291b8139 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:38.403222 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b03cbb-e5f3-436f-b793-9575291b8139 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:38.403913 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b03cbb-e5f3-436f-b793-9575291b8139 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:38.405283 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b03cbb-e5f3-436f-b793-9575291b8139 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:38.417853 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-18b03cbb-e5f3-436f-b793-9575291b8139 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 16:55:38.417853 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:38.417853 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 16:55:38.417853 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 16:55:38.417853 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 16:55:38.417853 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 16:55:38.417853 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 16:55:38.417853 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:38.417853 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 16:55:38.417853 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:38.417853 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 16:55:38.417853 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:38.417853 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:38.417853 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:38.417853 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:38.417853 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:38.417853 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:38.418856 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:38.418856 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:38.418856 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:38.418856 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:38.418856 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:38.418856 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:38.418856 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 16:55:38.418856 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 16:55:38.418856 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 16:55:38.418856 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 16:55:38.418856 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:38.418856 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:38.418856 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:38.418856 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:38.418856 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:38.418856 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:38.419696 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:38.419696 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:38.419696 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:38.419696 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:38.419696 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:38.419696 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:38.419696 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:38.419696 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:38.419696 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:38.419696 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:38.419696 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 16:55:38.419696 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:38.419696 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 16:55:38.419696 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:38.419696 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 498/969] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Jul 6 16:55:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:38.426444 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b03cbb-e5f3-436f-b793-9575291b8139 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:38.470725 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 472/970] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 16:55:38.493321 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d299c38-0b03-4839-9c3e-f41804667d1f None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:38.542382 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b03cbb-e5f3-436f-b793-9575291b8139 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:38.578869 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2739afa-a80e-4fad-a9dd-b41a44e299f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:38.579885 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2739afa-a80e-4fad-a9dd-b41a44e299f2 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:38.580180 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2739afa-a80e-4fad-a9dd-b41a44e299f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:38.580463 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2739afa-a80e-4fad-a9dd-b41a44e299f2 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:38.581174 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2739afa-a80e-4fad-a9dd-b41a44e299f2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:38.581486 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2739afa-a80e-4fad-a9dd-b41a44e299f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:38.582849 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2739afa-a80e-4fad-a9dd-b41a44e299f2 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:38.587970 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 499/971] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Jul 6 16:55:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:38.593485 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2739afa-a80e-4fad-a9dd-b41a44e299f2 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:38.603455 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:38.608394 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d299c38-0b03-4839-9c3e-f41804667d1f None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:38.641713 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:38.651608 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c340130d-7362-4073-ac64-38178d23b451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:38.652412 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340130d-7362-4073-ac64-38178d23b451 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:38.652700 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340130d-7362-4073-ac64-38178d23b451 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:38.652987 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c340130d-7362-4073-ac64-38178d23b451 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:38.663609 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c340130d-7362-4073-ac64-38178d23b451 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:55:38.672126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-c340130d-7362-4073-ac64-38178d23b451 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 06 16:55:38.672126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:38.672126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:38.672126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:38.672126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:38.672126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:38.672126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:38.672126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:38.672126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:38.672126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:38.672126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:38.672126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:38.672126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:55:38.672126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:55:38.672126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:55:38.672126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:55:38.673265 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:55:38.673265 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:55:38.673265 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:55:38.673265 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:55:38.673265 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:38.673265 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:38.673265 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:38.673265 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:38.673265 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:38.673265 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:38.673265 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:38.673265 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:38.673265 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:38.673265 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:38.673265 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:38.673265 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:38.676800 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:38.676800 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:38.676800 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:38.676800 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:38.676800 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:55:38.676800 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:55:38.676800 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:55:38.676800 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:55:38.676800 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:55:38.676800 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:55:38.676800 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 06 16:55:38.676800 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:38.676800 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 473/972] 10.222.0.40 () {56 vars in 1120 bytes} [Wed Jul 6 16:55:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:55:38.679606 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c340130d-7362-4073-ac64-38178d23b451 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:38.707016 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2739afa-a80e-4fad-a9dd-b41a44e299f2 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:38.740593 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c161136-6d33-4353-ba0c-d0ba8afbe8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:38.741825 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c161136-6d33-4353-ba0c-d0ba8afbe8a2 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:38.742106 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c161136-6d33-4353-ba0c-d0ba8afbe8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:38.742386 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c161136-6d33-4353-ba0c-d0ba8afbe8a2 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:38.743081 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c161136-6d33-4353-ba0c-d0ba8afbe8a2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:38.743383 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c161136-6d33-4353-ba0c-d0ba8afbe8a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:38.744792 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c161136-6d33-4353-ba0c-d0ba8afbe8a2 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:38.749821 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 500/973] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Jul 6 16:55:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:38.754625 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484ab51-8a7e-429e-8465-54e682a3f6e7 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:38.754625 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484ab51-8a7e-429e-8465-54e682a3f6e7 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:38.754625 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a484ab51-8a7e-429e-8465-54e682a3f6e7 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:38.754625 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 501/974] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:38.755687 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a484ab51-8a7e-429e-8465-54e682a3f6e7 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:38.790370 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c340130d-7362-4073-ac64-38178d23b451 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:38.824354 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b7b4f-6805-422e-93e5-c8db2adcfe09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:38.825027 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b7b4f-6805-422e-93e5-c8db2adcfe09 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:38.825313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b7b4f-6805-422e-93e5-c8db2adcfe09 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:38.825587 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b7b4f-6805-422e-93e5-c8db2adcfe09 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:38.826330 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459b7b4f-6805-422e-93e5-c8db2adcfe09 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:38.826636 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459b7b4f-6805-422e-93e5-c8db2adcfe09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:38.827116 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:38.828125 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459b7b4f-6805-422e-93e5-c8db2adcfe09 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:38.836515 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 474/975] 10.222.0.40 () {56 vars in 1115 bytes} [Wed Jul 6 16:55:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:38.836515 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e585e8-cf3c-4962-b375-cac2986cb2c6 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:38.836515 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e585e8-cf3c-4962-b375-cac2986cb2c6 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:38.838429 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e585e8-cf3c-4962-b375-cac2986cb2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:38.856497 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-91e585e8-cf3c-4962-b375-cac2986cb2c6 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:38.857666 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-91e585e8-cf3c-4962-b375-cac2986cb2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:38.861494 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a484ab51-8a7e-429e-8465-54e682a3f6e7 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:38.895097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639729a1-9ab1-4d67-83e1-e6793646a57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:38.895775 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639729a1-9ab1-4d67-83e1-e6793646a57b None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:38.896314 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639729a1-9ab1-4d67-83e1-e6793646a57b None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:38.896594 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639729a1-9ab1-4d67-83e1-e6793646a57b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:38.897910 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639729a1-9ab1-4d67-83e1-e6793646a57b None admin] RBAC: Adding query filter params (interface=admin, service_id=ee7f1a7143e54e258d1e65d1b0c62a09, region_id=RegionOne) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:38.898223 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639729a1-9ab1-4d67-83e1-e6793646a57b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:38.899571 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639729a1-9ab1-4d67-83e1-e6793646a57b None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:38.901115 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:38.904808 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 502/976] 10.222.0.40 () {56 vars in 1249 bytes} [Wed Jul 6 16:55:38 2022] GET /identity/v3/endpoints?service_id=ee7f1a7143e54e258d1e65d1b0c62a09&interface=admin®ion_id=RegionOne => generated 193 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:38.905516 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639729a1-9ab1-4d67-83e1-e6793646a57b None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:38.925148 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 475/977] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:55:38.925973 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e585e8-cf3c-4962-b375-cac2986cb2c6 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:39.006557 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639729a1-9ab1-4d67-83e1-e6793646a57b None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:39.025760 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e585e8-cf3c-4962-b375-cac2986cb2c6 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:39.040417 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5f04b1-e32e-4b79-b4dc-b3a657797647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:39.041212 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f04b1-e32e-4b79-b4dc-b3a657797647 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:39.041501 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f04b1-e32e-4b79-b4dc-b3a657797647 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:39.041781 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f04b1-e32e-4b79-b4dc-b3a657797647 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:39.043663 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:39.052757 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5f04b1-e32e-4b79-b4dc-b3a657797647 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:39.054223 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5f04b1-e32e-4b79-b4dc-b3a657797647 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:39.059152 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2d3461-9136-4906-a515-2395ae02f7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:39.059849 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d3461-9136-4906-a515-2395ae02f7c2 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:39.060132 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d3461-9136-4906-a515-2395ae02f7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:39.060405 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d3461-9136-4906-a515-2395ae02f7c2 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:39.061077 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2d3461-9136-4906-a515-2395ae02f7c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:39.062423 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2d3461-9136-4906-a515-2395ae02f7c2 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:39.068508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 476/978] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Jul 6 16:55:38 2022] GET /identity/v3/services => generated 1497 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 16:55:39.069099 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2d3461-9136-4906-a515-2395ae02f7c2 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:39.071570 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-4a5f04b1-e32e-4b79-b4dc-b3a657797647 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:39.071570 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:39.071570 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:39.071570 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:39.071570 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:39.071570 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:39.071570 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:39.071570 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:39.071570 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:39.071570 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:39.071570 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:39.071570 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:39.071570 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:55:39.071570 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:55:39.071570 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:55:39.072564 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:55:39.072564 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:39.072564 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:39.072564 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:39.072564 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:39.072564 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:39.072564 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:39.072564 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:39.072564 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:39.072564 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:39.072564 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:39.072564 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:39.072564 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:39.072564 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:39.072564 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:39.072564 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:39.072564 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:39.073490 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:55:39.073490 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:55:39.073490 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:39.073490 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:39.076177 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 503/979] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:39.078943 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5f04b1-e32e-4b79-b4dc-b3a657797647 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:39.128188 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:39.193092 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5f04b1-e32e-4b79-b4dc-b3a657797647 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:39.193288 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2d3461-9136-4906-a515-2395ae02f7c2 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:39.229976 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59d760-1975-436b-bf2b-f86eef8d9112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:39.230608 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59d760-1975-436b-bf2b-f86eef8d9112 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:39.230892 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59d760-1975-436b-bf2b-f86eef8d9112 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:39.231799 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d59d760-1975-436b-bf2b-f86eef8d9112 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:39.231904 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59d760-1975-436b-bf2b-f86eef8d9112 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:39.232913 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59d760-1975-436b-bf2b-f86eef8d9112 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:39.233571 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d59d760-1975-436b-bf2b-f86eef8d9112 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:39.235698 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5c49f2-fe74-4783-99b4-6d5d1a85a9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:39.236942 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c49f2-fe74-4783-99b4-6d5d1a85a9d8 None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:39.237229 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c49f2-fe74-4783-99b4-6d5d1a85a9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:39.238540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5c49f2-fe74-4783-99b4-6d5d1a85a9d8 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:39.238540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5c49f2-fe74-4783-99b4-6d5d1a85a9d8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:39.240281 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5c49f2-fe74-4783-99b4-6d5d1a85a9d8 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:39.246108 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 504/980] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:55:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:39.251786 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59d760-1975-436b-bf2b-f86eef8d9112 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:39.297895 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:39.349948 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d59d760-1975-436b-bf2b-f86eef8d9112 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:39.365875 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 477/981] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Jul 6 16:55:39 2022] POST /identity/v3/services => generated 242 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 16:55:39.392893 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79223cfb-32dc-4b94-863d-655eb81a89b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:39.393742 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79223cfb-32dc-4b94-863d-655eb81a89b8 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:39.394081 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79223cfb-32dc-4b94-863d-655eb81a89b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:39.394361 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79223cfb-32dc-4b94-863d-655eb81a89b8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:39.404285 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79223cfb-32dc-4b94-863d-655eb81a89b8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:39.405947 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79223cfb-32dc-4b94-863d-655eb81a89b8 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:39.422823 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-79223cfb-32dc-4b94-863d-655eb81a89b8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 06 16:55:39.422823 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:39.422823 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:39.422823 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:39.422823 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:39.422823 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:39.422823 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:39.422823 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:39.422823 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:39.422823 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:39.422823 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:39.422823 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:39.422823 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 16:55:39.422823 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 16:55:39.422823 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 16:55:39.422823 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 16:55:39.423892 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:39.423892 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:39.423892 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 16:55:39.423892 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:39.423892 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 16:55:39.423892 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:39.423892 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:39.423892 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:39.423892 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:39.423892 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:39.423892 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:39.423892 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:39.423892 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:39.423892 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:39.423892 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:39.423892 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:39.423892 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:39.423892 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:39.425077 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:39.425077 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:39.425077 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 16:55:39.425077 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 16:55:39.425077 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 16:55:39.425077 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 16:55:39.425077 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 16:55:39.425077 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 16:55:39.425077 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 06 16:55:39.425077 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:39.425077 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 505/982] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Jul 6 16:55:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:55:39.430091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5c49f2-fe74-4783-99b4-6d5d1a85a9d8 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:39.542599 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5c49f2-fe74-4783-99b4-6d5d1a85a9d8 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:39.544133 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:39.578704 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2804d5a-7858-4f78-a222-fd2468db6447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:39.579384 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2804d5a-7858-4f78-a222-fd2468db6447 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:39.579671 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2804d5a-7858-4f78-a222-fd2468db6447 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:39.579977 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2804d5a-7858-4f78-a222-fd2468db6447 None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:39.580843 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2804d5a-7858-4f78-a222-fd2468db6447 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:39.581224 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2804d5a-7858-4f78-a222-fd2468db6447 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:39.582611 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2804d5a-7858-4f78-a222-fd2468db6447 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:39.600143 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 478/983] 10.222.0.40 () {56 vars in 1149 bytes} [Wed Jul 6 16:55:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:39.606138 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79223cfb-32dc-4b94-863d-655eb81a89b8 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:39.707300 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79223cfb-32dc-4b94-863d-655eb81a89b8 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:39.708888 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:39.754077 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa9a520-ce41-44c1-b0c8-ec3819484ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:39.754739 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa9a520-ce41-44c1-b0c8-ec3819484ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:39.755024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa9a520-ce41-44c1-b0c8-ec3819484ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:39.755304 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa9a520-ce41-44c1-b0c8-ec3819484ef7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:39.825417 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa9a520-ce41-44c1-b0c8-ec3819484ef7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:39.826803 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa9a520-ce41-44c1-b0c8-ec3819484ef7 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:39.930133 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 506/984] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:55:39 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 06 16:55:39.931619 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2804d5a-7858-4f78-a222-fd2468db6447 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:40.030770 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2804d5a-7858-4f78-a222-fd2468db6447 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:40.064823 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb179b-b062-48bb-b4f1-2cedee4d3b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:40.065620 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb179b-b062-48bb-b4f1-2cedee4d3b0e None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:40.065904 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb179b-b062-48bb-b4f1-2cedee4d3b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:40.066179 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bb179b-b062-48bb-b4f1-2cedee4d3b0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:40.076436 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bb179b-b062-48bb-b4f1-2cedee4d3b0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:40.077820 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bb179b-b062-48bb-b4f1-2cedee4d3b0e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:40.095296 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-17bb179b-b062-48bb-b4f1-2cedee4d3b0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:40.095296 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:40.095296 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:40.095296 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:40.095296 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:40.095296 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:40.095296 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:40.095296 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:40.095296 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:40.095296 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:40.095296 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:40.095296 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:40.095296 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:55:40.095296 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:55:40.095296 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:55:40.096244 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:55:40.096244 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:40.096244 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:40.096244 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:40.096244 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:40.096244 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:40.096244 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:40.096244 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:40.096244 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:40.096244 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:40.096244 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:40.096244 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:40.096244 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:40.096244 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:40.096244 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:40.096244 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:40.096244 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:40.097149 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:55:40.097149 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:55:40.097149 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:40.097149 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:40.097149 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 479/985] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:55:40.102753 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa9a520-ce41-44c1-b0c8-ec3819484ef7 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:40.104946 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:40.209692 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa9a520-ce41-44c1-b0c8-ec3819484ef7 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:40.243331 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b4ce3d-a300-4d66-98ad-6d661c927dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:40.244023 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4ce3d-a300-4d66-98ad-6d661c927dff None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:40.244312 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4ce3d-a300-4d66-98ad-6d661c927dff None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:40.244592 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b4ce3d-a300-4d66-98ad-6d661c927dff None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:40.245317 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b4ce3d-a300-4d66-98ad-6d661c927dff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:40.245621 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b4ce3d-a300-4d66-98ad-6d661c927dff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:40.247046 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b4ce3d-a300-4d66-98ad-6d661c927dff None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:40.263769 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 507/986] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:55:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:40.264983 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb179b-b062-48bb-b4f1-2cedee4d3b0e None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:40.282192 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:40.370159 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bb179b-b062-48bb-b4f1-2cedee4d3b0e None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:40.403462 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd5cdb-d42c-4aff-9f20-64af34965eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:40.404291 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd5cdb-d42c-4aff-9f20-64af34965eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:40.404572 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd5cdb-d42c-4aff-9f20-64af34965eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:40.404850 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccd5cdb-d42c-4aff-9f20-64af34965eb1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:40.411412 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd5cdb-d42c-4aff-9f20-64af34965eb1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:40.413429 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccd5cdb-d42c-4aff-9f20-64af34965eb1 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:40.427382 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-4ccd5cdb-d42c-4aff-9f20-64af34965eb1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:40.427382 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:40.427382 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:40.427382 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:40.427382 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:40.427382 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:40.427382 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:40.427382 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:40.427382 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:40.427382 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:40.427382 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:40.427382 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:40.427382 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 16:55:40.427382 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 16:55:40.427382 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 16:55:40.428452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 16:55:40.428452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:40.428452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:40.428452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:40.428452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:40.428452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:40.428452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:40.428452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:40.428452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:40.428452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:40.428452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:40.428452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:40.428452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:40.428452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:40.428452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:40.428452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:40.428452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:40.434276 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 16:55:40.434276 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 16:55:40.434276 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 16:55:40.434276 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 16:55:40.434276 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 16:55:40.434276 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 16:55:40.434276 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:40.434276 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:40.434276 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 480/987] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:55:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:40.434760 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b4ce3d-a300-4d66-98ad-6d661c927dff None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:40.497923 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:40.537596 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b4ce3d-a300-4d66-98ad-6d661c927dff None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:40.571271 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57910f6a-e5b6-4853-b948-3f375463ea38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:40.571969 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57910f6a-e5b6-4853-b948-3f375463ea38 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:40.572259 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57910f6a-e5b6-4853-b948-3f375463ea38 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:40.572542 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57910f6a-e5b6-4853-b948-3f375463ea38 None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:40.573327 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57910f6a-e5b6-4853-b948-3f375463ea38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:40.573641 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57910f6a-e5b6-4853-b948-3f375463ea38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:40.575074 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57910f6a-e5b6-4853-b948-3f375463ea38 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:40.588012 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 508/988] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 6 16:55:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:40.593856 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd5cdb-d42c-4aff-9f20-64af34965eb1 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:40.664868 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:40.694945 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccd5cdb-d42c-4aff-9f20-64af34965eb1 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:40.732813 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01388146-795d-4b63-a5a6-f8561785ece8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:40.733880 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01388146-795d-4b63-a5a6-f8561785ece8 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:40.733880 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01388146-795d-4b63-a5a6-f8561785ece8 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:40.734051 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01388146-795d-4b63-a5a6-f8561785ece8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:40.734845 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01388146-795d-4b63-a5a6-f8561785ece8 None admin] RBAC: Adding query filter params (role.id=4d3a77c1547c43aba11b60377d04d537, scope.project.id=7938b0346d4240ab89774327d09a89ae, user.id=1905e6f8c3e24e3cbfefce2827baa370) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:40.735160 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01388146-795d-4b63-a5a6-f8561785ece8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:40.736631 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01388146-795d-4b63-a5a6-f8561785ece8 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:40.743128 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 481/989] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:55:40 2022] GET /identity/v3/role_assignments?user.id=1905e6f8c3e24e3cbfefce2827baa370&scope.project.id=7938b0346d4240ab89774327d09a89ae&role.id=4d3a77c1547c43aba11b60377d04d537 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:41.122183 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1417fa6-6d24-4223-bab5-7fbd1d725034 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:41.122369 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1417fa6-6d24-4223-bab5-7fbd1d725034 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:41.122659 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1417fa6-6d24-4223-bab5-7fbd1d725034 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:41.124064 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 509/990] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:41.129165 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bcfe45-22a7-417d-ba49-de55ab5d8197 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:41.129462 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bcfe45-22a7-417d-ba49-de55ab5d8197 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:41.129746 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bcfe45-22a7-417d-ba49-de55ab5d8197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:41.146471 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-28bcfe45-22a7-417d-ba49-de55ab5d8197 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:41.148203 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-28bcfe45-22a7-417d-ba49-de55ab5d8197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:41.219603 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 482/991] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:41 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:55:41.242208 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1417fa6-6d24-4223-bab5-7fbd1d725034 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:41.344679 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1417fa6-6d24-4223-bab5-7fbd1d725034 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:41.367153 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:41.378951 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca2cec-0861-451d-905c-d49425eae691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:41.379684 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca2cec-0861-451d-905c-d49425eae691 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:41.379999 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca2cec-0861-451d-905c-d49425eae691 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:41.380274 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ca2cec-0861-451d-905c-d49425eae691 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:41.380914 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca2cec-0861-451d-905c-d49425eae691 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:41.382886 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ca2cec-0861-451d-905c-d49425eae691 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:41.394772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-f5ca2cec-0861-451d-905c-d49425eae691 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 16:55:41.394772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:41.394772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 16:55:41.394772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 16:55:41.394772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 16:55:41.394772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 16:55:41.394772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 16:55:41.394772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:41.394772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 16:55:41.394772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:41.394772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 16:55:41.394772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:41.394772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:41.394772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:41.394772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:41.394772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:41.394772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:41.395783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:41.395783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:41.395783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:41.395783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:41.395783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:41.395783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:41.395783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 16:55:41.395783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 16:55:41.395783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 16:55:41.395783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 16:55:41.395783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:41.395783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:41.395783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:41.395783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:41.395783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:41.395783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:41.397549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:41.397549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:41.397549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:41.397549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:41.397549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:41.397549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:41.397549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:41.397549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:41.397549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:41.397549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:41.397549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 16:55:41.397549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:41.397549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 06 16:55:41.397549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:41.397549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 510/992] 10.222.0.40 () {56 vars in 1126 bytes} [Wed Jul 6 16:55:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:41.402844 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa0e8b-0721-438c-9b27-647133e27a19 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:41.403574 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa0e8b-0721-438c-9b27-647133e27a19 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:41.403574 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fa0e8b-0721-438c-9b27-647133e27a19 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:41.404261 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca2cec-0861-451d-905c-d49425eae691 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:41.405374 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 483/993] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:41.410090 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334be933-6e42-4c02-aa93-6524ca0d88f9 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:41.410378 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334be933-6e42-4c02-aa93-6524ca0d88f9 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:41.411579 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334be933-6e42-4c02-aa93-6524ca0d88f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:41.428782 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-334be933-6e42-4c02-aa93-6524ca0d88f9 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:41.429959 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-334be933-6e42-4c02-aa93-6524ca0d88f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:41.494833 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:41.502687 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 484/994] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:41 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 16:55:41.514849 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ca2cec-0861-451d-905c-d49425eae691 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:41.525112 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334be933-6e42-4c02-aa93-6524ca0d88f9 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:41.555349 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e981b-4a54-4869-88f5-573f8f0f799d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:41.556195 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e981b-4a54-4869-88f5-573f8f0f799d None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:41.556488 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e981b-4a54-4869-88f5-573f8f0f799d None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:41.556771 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353e981b-4a54-4869-88f5-573f8f0f799d None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:41.558016 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e981b-4a54-4869-88f5-573f8f0f799d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:41.558326 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e981b-4a54-4869-88f5-573f8f0f799d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:41.559853 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353e981b-4a54-4869-88f5-573f8f0f799d None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:41.566111 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 511/995] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Jul 6 16:55:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:41.571023 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e981b-4a54-4869-88f5-573f8f0f799d None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:41.624076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:41.629626 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334be933-6e42-4c02-aa93-6524ca0d88f9 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:41.664809 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25524c1-8468-4ffe-b421-80886743ea80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:41.665606 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25524c1-8468-4ffe-b421-80886743ea80 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:41.665884 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25524c1-8468-4ffe-b421-80886743ea80 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:41.666155 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25524c1-8468-4ffe-b421-80886743ea80 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:41.666796 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25524c1-8468-4ffe-b421-80886743ea80 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:41.668180 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25524c1-8468-4ffe-b421-80886743ea80 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:41.676637 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353e981b-4a54-4869-88f5-573f8f0f799d None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:41.679621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-d25524c1-8468-4ffe-b421-80886743ea80 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 06 16:55:41.679621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:41.679621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 16:55:41.679621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 16:55:41.679621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 16:55:41.679621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 16:55:41.679621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 16:55:41.679621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:41.679621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 06 16:55:41.679621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:41.679621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 16:55:41.679621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:41.679621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:41.679621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:41.679621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:41.679621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:41.679621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:41.680602 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:41.680602 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:41.680602 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:41.680602 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:41.680602 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:41.680602 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:41.680602 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 16:55:41.680602 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 16:55:41.680602 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 16:55:41.680602 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 16:55:41.680602 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:41.680602 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:41.680602 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:41.680602 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:41.680602 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:41.680602 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:41.681456 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:41.681456 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:41.681456 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:41.681456 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:41.681456 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:41.681456 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:41.681456 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:41.681456 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:41.681456 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:41.681456 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:41.681456 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 16:55:41.681456 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:41.681456 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 06 16:55:41.681456 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:41.685672 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 485/996] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:55:41 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:55:41.686477 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25524c1-8468-4ffe-b421-80886743ea80 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:41.712330 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f449616-dc31-4432-ad47-0f7a25a545ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:41.712999 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f449616-dc31-4432-ad47-0f7a25a545ae None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:41.713286 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f449616-dc31-4432-ad47-0f7a25a545ae None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:41.713567 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f449616-dc31-4432-ad47-0f7a25a545ae None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:41.714385 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f449616-dc31-4432-ad47-0f7a25a545ae None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:41.714700 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f449616-dc31-4432-ad47-0f7a25a545ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:41.718644 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f449616-dc31-4432-ad47-0f7a25a545ae None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:41.722999 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 512/997] 10.222.0.40 () {56 vars in 1122 bytes} [Wed Jul 6 16:55:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:41.729172 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f449616-dc31-4432-ad47-0f7a25a545ae None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:41.752926 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:41.800136 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25524c1-8468-4ffe-b421-80886743ea80 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:41.835026 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d8c2e1-0176-49f1-a444-7d1d54a86fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:41.835741 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d8c2e1-0176-49f1-a444-7d1d54a86fea None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:41.836030 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d8c2e1-0176-49f1-a444-7d1d54a86fea None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:41.836313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d8c2e1-0176-49f1-a444-7d1d54a86fea None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:41.837015 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d8c2e1-0176-49f1-a444-7d1d54a86fea None admin] RBAC: Adding query filter params (name=compute) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:41.837321 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d8c2e1-0176-49f1-a444-7d1d54a86fea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:41.838678 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d8c2e1-0176-49f1-a444-7d1d54a86fea None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:41.839392 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f449616-dc31-4432-ad47-0f7a25a545ae None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:41.848320 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 486/998] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Jul 6 16:55:41 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:41.848975 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d8c2e1-0176-49f1-a444-7d1d54a86fea None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:41.855482 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:41.876448 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4eb6c-327d-47dd-822e-958d4151853c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:41.877119 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4eb6c-327d-47dd-822e-958d4151853c None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:41.877408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4eb6c-327d-47dd-822e-958d4151853c None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:41.877761 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe4eb6c-327d-47dd-822e-958d4151853c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:41.878618 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe4eb6c-327d-47dd-822e-958d4151853c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:41.879980 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe4eb6c-327d-47dd-822e-958d4151853c None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:41.950451 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d8c2e1-0176-49f1-a444-7d1d54a86fea None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:41.971759 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 513/999] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 6 16:55:41 2022] POST /identity/v3/endpoints => generated 325 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 16:55:41.981113 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:42.000057 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d68a16d-1048-4074-be92-02b6adaee052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:42.000732 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d68a16d-1048-4074-be92-02b6adaee052 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:42.001012 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d68a16d-1048-4074-be92-02b6adaee052 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:42.001319 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d68a16d-1048-4074-be92-02b6adaee052 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:42.001991 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d68a16d-1048-4074-be92-02b6adaee052 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:42.002288 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d68a16d-1048-4074-be92-02b6adaee052 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:42.003666 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d68a16d-1048-4074-be92-02b6adaee052 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:42.010258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 487/1000] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Jul 6 16:55:41 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:42.016236 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4eb6c-327d-47dd-822e-958d4151853c None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:42.125404 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe4eb6c-327d-47dd-822e-958d4151853c None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:42.167090 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d80a9d-3a3e-4341-89e0-5017dd0bb4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:42.167791 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d80a9d-3a3e-4341-89e0-5017dd0bb4c1 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:42.168077 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d80a9d-3a3e-4341-89e0-5017dd0bb4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:42.168351 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d80a9d-3a3e-4341-89e0-5017dd0bb4c1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:42.169140 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d80a9d-3a3e-4341-89e0-5017dd0bb4c1 None admin] RBAC: Adding query filter params (interface=public, service_id=1becc81dbb3543c9bc2888f588f5097d, region_id=RegionOne) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:42.169450 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d80a9d-3a3e-4341-89e0-5017dd0bb4c1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:42.170814 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d80a9d-3a3e-4341-89e0-5017dd0bb4c1 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:42.176243 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 514/1001] 10.222.0.40 () {56 vars in 1251 bytes} [Wed Jul 6 16:55:42 2022] GET /identity/v3/endpoints?service_id=1becc81dbb3543c9bc2888f588f5097d&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:42.182160 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d68a16d-1048-4074-be92-02b6adaee052 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:42.283863 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d68a16d-1048-4074-be92-02b6adaee052 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:42.319324 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc3cb4-6b2c-42a8-9ff3-98353b1a9681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:42.320127 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc3cb4-6b2c-42a8-9ff3-98353b1a9681 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:42.320422 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc3cb4-6b2c-42a8-9ff3-98353b1a9681 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:42.320701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc3cb4-6b2c-42a8-9ff3-98353b1a9681 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:42.321385 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfc3cb4-6b2c-42a8-9ff3-98353b1a9681 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:42.322752 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfc3cb4-6b2c-42a8-9ff3-98353b1a9681 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:42.335775 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 488/1002] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Jul 6 16:55:42 2022] GET /identity/v3/services => generated 1727 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 16:55:42.825303 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ced734-1578-411f-9c12-7773613e893d None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:42.825842 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ced734-1578-411f-9c12-7773613e893d None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:42.825939 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ced734-1578-411f-9c12-7773613e893d None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:42.827262 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 515/1003] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:42.832333 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461e03f-7b3c-4a52-bd25-96182ab981b5 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:42.832715 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461e03f-7b3c-4a52-bd25-96182ab981b5 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:42.832905 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461e03f-7b3c-4a52-bd25-96182ab981b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:42.849372 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-9461e03f-7b3c-4a52-bd25-96182ab981b5 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:42.851098 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-9461e03f-7b3c-4a52-bd25-96182ab981b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:42.894174 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:42.917804 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 489/1004] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:42 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:55:42.940546 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ced734-1578-411f-9c12-7773613e893d None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:42.983631 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:43.045461 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ced734-1578-411f-9c12-7773613e893d None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:43.080672 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af22ab7-6b16-4fd7-a21e-dc8c224c9fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:43.081492 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af22ab7-6b16-4fd7-a21e-dc8c224c9fa7 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:43.081781 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af22ab7-6b16-4fd7-a21e-dc8c224c9fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:43.082058 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af22ab7-6b16-4fd7-a21e-dc8c224c9fa7 None admin] PATH_INFO: `/v3/services/image` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:43.082728 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af22ab7-6b16-4fd7-a21e-dc8c224c9fa7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:43.084186 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af22ab7-6b16-4fd7-a21e-dc8c224c9fa7 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:43.096715 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-1af22ab7-6b16-4fd7-a21e-dc8c224c9fa7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 06 16:55:43.096715 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:43.096715 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 16:55:43.096715 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 16:55:43.096715 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 16:55:43.096715 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 16:55:43.096715 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 16:55:43.096715 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:43.096715 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 06 16:55:43.096715 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:43.096715 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 16:55:43.096715 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:43.096715 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:43.096715 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:43.096715 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:43.096715 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:43.096715 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:43.097784 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:43.097784 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:43.097784 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:43.097784 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:43.097784 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:43.097784 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:43.097784 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 16:55:43.097784 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 16:55:43.097784 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 16:55:43.097784 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 16:55:43.097784 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:43.097784 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:43.097784 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:43.097784 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:43.097784 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:43.097784 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:43.104178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:43.104178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:43.104178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:43.104178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:43.104178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:43.104178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:43.104178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:43.104178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:43.104178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:43.104178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:43.104178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 16:55:43.104178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:43.104178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 06 16:55:43.104178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:43.104178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 516/1005] 10.222.0.40 () {56 vars in 1105 bytes} [Wed Jul 6 16:55:42 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:43.106621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461e03f-7b3c-4a52-bd25-96182ab981b5 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:43.205217 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461e03f-7b3c-4a52-bd25-96182ab981b5 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:43.248366 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f94d3dd-8b94-4c72-9f09-0f5a471bc6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:43.248366 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f94d3dd-8b94-4c72-9f09-0f5a471bc6e3 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:43.248366 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f94d3dd-8b94-4c72-9f09-0f5a471bc6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:43.248366 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f94d3dd-8b94-4c72-9f09-0f5a471bc6e3 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:43.248734 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f94d3dd-8b94-4c72-9f09-0f5a471bc6e3 None admin] RBAC: Adding query filter params (name=image) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:43.248734 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f94d3dd-8b94-4c72-9f09-0f5a471bc6e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:43.248734 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f94d3dd-8b94-4c72-9f09-0f5a471bc6e3 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:43.248734 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 490/1006] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:43 2022] GET /identity/v3/services?name=image => generated 122 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:43.252826 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af22ab7-6b16-4fd7-a21e-dc8c224c9fa7 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:43.353811 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af22ab7-6b16-4fd7-a21e-dc8c224c9fa7 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:43.355344 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:43.389016 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6cd04-65ea-4c3b-81e7-33d0e1d01a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:43.389695 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6cd04-65ea-4c3b-81e7-33d0e1d01a89 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:43.389983 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6cd04-65ea-4c3b-81e7-33d0e1d01a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:43.390264 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6cd04-65ea-4c3b-81e7-33d0e1d01a89 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:43.391079 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb6cd04-65ea-4c3b-81e7-33d0e1d01a89 None admin] RBAC: Adding query filter params (type=image) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:43.391713 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb6cd04-65ea-4c3b-81e7-33d0e1d01a89 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:43.393547 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb6cd04-65ea-4c3b-81e7-33d0e1d01a89 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:43.398940 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 517/1007] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:43 2022] GET /identity/v3/services?type=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:44.000407 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85049964-6754-425a-a386-5f654928f699 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:44.000906 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85049964-6754-425a-a386-5f654928f699 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:44.000906 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85049964-6754-425a-a386-5f654928f699 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:44.002407 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 491/1008] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:44.007973 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d98e5-be45-4068-9ed9-14cd0e9d5a62 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:44.008375 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d98e5-be45-4068-9ed9-14cd0e9d5a62 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:44.008570 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6d98e5-be45-4068-9ed9-14cd0e9d5a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:44.025204 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-cb6d98e5-be45-4068-9ed9-14cd0e9d5a62 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:44.026379 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-cb6d98e5-be45-4068-9ed9-14cd0e9d5a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:44.094946 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 518/1009] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:44 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:55:44.118164 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85049964-6754-425a-a386-5f654928f699 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:44.162981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:44.218742 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85049964-6754-425a-a386-5f654928f699 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:44.252875 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe0a790-b104-47a9-aa21-8658b904f697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:44.253581 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0a790-b104-47a9-aa21-8658b904f697 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:44.253864 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0a790-b104-47a9-aa21-8658b904f697 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:44.254141 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe0a790-b104-47a9-aa21-8658b904f697 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:44.255301 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe0a790-b104-47a9-aa21-8658b904f697 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:44.257302 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe0a790-b104-47a9-aa21-8658b904f697 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:44.259117 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 492/1010] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:44.264650 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d98e5-be45-4068-9ed9-14cd0e9d5a62 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:44.359301 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:44.365477 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6d98e5-be45-4068-9ed9-14cd0e9d5a62 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:44.371621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0156bee-7114-4b8d-a462-6af645244533 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:44.371946 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0156bee-7114-4b8d-a462-6af645244533 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:44.372233 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0156bee-7114-4b8d-a462-6af645244533 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:44.373877 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 493/1011] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:44.378612 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bce6e0-8234-4ed0-b817-ae8def0a9375 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:44.378905 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bce6e0-8234-4ed0-b817-ae8def0a9375 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:44.379188 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bce6e0-8234-4ed0-b817-ae8def0a9375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:44.395498 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-20bce6e0-8234-4ed0-b817-ae8def0a9375 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:44.397026 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-20bce6e0-8234-4ed0-b817-ae8def0a9375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:44.401819 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558ea5e6-869f-427f-91c6-be1dcbe9461c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:44.402507 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ea5e6-869f-427f-91c6-be1dcbe9461c None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:44.402792 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ea5e6-869f-427f-91c6-be1dcbe9461c None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:44.403071 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558ea5e6-869f-427f-91c6-be1dcbe9461c None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:44.403803 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558ea5e6-869f-427f-91c6-be1dcbe9461c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:44.405678 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558ea5e6-869f-427f-91c6-be1dcbe9461c None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:44.464747 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 494/1012] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:44 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 16:55:44.487497 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bce6e0-8234-4ed0-b817-ae8def0a9375 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:44.515387 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 519/1013] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Jul 6 16:55:44 2022] POST /identity/v3/projects => generated 311 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 16:55:44.544064 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:44.605927 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bce6e0-8234-4ed0-b817-ae8def0a9375 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:44.644413 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dace91-cd87-4f6b-a225-1b6432c2ad01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:44.645242 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dace91-cd87-4f6b-a225-1b6432c2ad01 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:44.645508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dace91-cd87-4f6b-a225-1b6432c2ad01 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:44.645779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dace91-cd87-4f6b-a225-1b6432c2ad01 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:44.646716 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dace91-cd87-4f6b-a225-1b6432c2ad01 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:44.648154 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dace91-cd87-4f6b-a225-1b6432c2ad01 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:44.660703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-23dace91-cd87-4f6b-a225-1b6432c2ad01 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 06 16:55:44.660703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:44.660703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 16:55:44.660703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 16:55:44.660703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 16:55:44.660703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 16:55:44.660703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 16:55:44.660703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:44.660703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 06 16:55:44.660703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:44.660703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 16:55:44.660703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:44.660703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:44.660703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:44.660703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:44.660703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:44.660703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:44.661796 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:44.661796 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:44.661796 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:44.661796 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:44.661796 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:44.661796 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:44.661796 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 16:55:44.661796 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 16:55:44.661796 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 16:55:44.661796 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 16:55:44.661796 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:44.661796 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:44.661796 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:44.661796 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:44.661796 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:44.661796 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:44.663175 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:44.663175 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:44.663175 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:44.663175 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:44.663175 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:44.663175 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:44.663175 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:44.663175 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:44.663175 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:44.663175 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:44.663175 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 16:55:44.663175 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:44.663175 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 06 16:55:44.663175 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:44.667505 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 495/1014] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:55:44 2022] GET /identity/v3/services/compute => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:55:44.671867 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558ea5e6-869f-427f-91c6-be1dcbe9461c None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:44.779488 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558ea5e6-869f-427f-91c6-be1dcbe9461c None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:44.812340 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7564ab9-685a-4129-a00c-ea3770037837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:44.813012 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7564ab9-685a-4129-a00c-ea3770037837 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:44.813296 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7564ab9-685a-4129-a00c-ea3770037837 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:44.813572 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7564ab9-685a-4129-a00c-ea3770037837 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:44.814290 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7564ab9-685a-4129-a00c-ea3770037837 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:44.814595 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7564ab9-685a-4129-a00c-ea3770037837 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:44.815986 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7564ab9-685a-4129-a00c-ea3770037837 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:44.821795 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 520/1015] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Jul 6 16:55:44 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:44.827666 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dace91-cd87-4f6b-a225-1b6432c2ad01 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:44.929436 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dace91-cd87-4f6b-a225-1b6432c2ad01 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:44.931801 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:44.963774 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6517f6-c2ae-4a31-a585-d610b04a0d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:44.964451 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6517f6-c2ae-4a31-a585-d610b04a0d46 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:44.964738 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6517f6-c2ae-4a31-a585-d610b04a0d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:44.965016 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6517f6-c2ae-4a31-a585-d610b04a0d46 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:44.965719 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6517f6-c2ae-4a31-a585-d610b04a0d46 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:44.966028 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6517f6-c2ae-4a31-a585-d610b04a0d46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:44.967394 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6517f6-c2ae-4a31-a585-d610b04a0d46 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:44.977117 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 496/1016] 10.222.0.40 () {56 vars in 1112 bytes} [Wed Jul 6 16:55:44 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:44.978354 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7564ab9-685a-4129-a00c-ea3770037837 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:44.994591 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:45.081351 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7564ab9-685a-4129-a00c-ea3770037837 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:45.114939 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3187fa-0aaa-468e-ba40-5f3a5145db97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:45.115637 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3187fa-0aaa-468e-ba40-5f3a5145db97 None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:45.115951 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3187fa-0aaa-468e-ba40-5f3a5145db97 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:45.116228 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3187fa-0aaa-468e-ba40-5f3a5145db97 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:45.116960 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3187fa-0aaa-468e-ba40-5f3a5145db97 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:45.118326 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3187fa-0aaa-468e-ba40-5f3a5145db97 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:45.219759 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 521/1017] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 6 16:55:44 2022] POST /identity/v3/endpoints => generated 334 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 16:55:45.419684 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfe4c93-caa4-4c6c-8fe9-e4f3ea7d0683 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:45.420277 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfe4c93-caa4-4c6c-8fe9-e4f3ea7d0683 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:45.420277 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfe4c93-caa4-4c6c-8fe9-e4f3ea7d0683 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:45.421643 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 497/1018] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:45.426698 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39976a-cf7d-4570-a17f-827f3005475e None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:45.426989 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39976a-cf7d-4570-a17f-827f3005475e None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:45.427272 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a39976a-cf7d-4570-a17f-827f3005475e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:45.444799 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-1a39976a-cf7d-4570-a17f-827f3005475e None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:45.445974 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-1a39976a-cf7d-4570-a17f-827f3005475e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:45.509864 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:45.522272 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 522/1019] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:45 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:55:45.544679 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfe4c93-caa4-4c6c-8fe9-e4f3ea7d0683 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:45.644955 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfe4c93-caa4-4c6c-8fe9-e4f3ea7d0683 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:45.681023 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eebc9ce-3238-4846-8365-fff424cd5917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:45.681620 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebc9ce-3238-4846-8365-fff424cd5917 None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:45.681905 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebc9ce-3238-4846-8365-fff424cd5917 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:45.682181 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebc9ce-3238-4846-8365-fff424cd5917 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:45.682911 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eebc9ce-3238-4846-8365-fff424cd5917 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:45.684310 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eebc9ce-3238-4846-8365-fff424cd5917 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:45.753478 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 498/1020] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Jul 6 16:55:45 2022] POST /identity/v3/services => generated 242 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 16:55:46.637426 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060795f3-6839-4c56-bf5a-3d0bf789f416 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:46.637960 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060795f3-6839-4c56-bf5a-3d0bf789f416 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:46.638039 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060795f3-6839-4c56-bf5a-3d0bf789f416 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:46.639504 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 523/1021] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:46.644485 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b157a-9f4a-419b-a6a8-262115efbc9b None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:46.644859 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b157a-9f4a-419b-a6a8-262115efbc9b None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:46.645058 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869b157a-9f4a-419b-a6a8-262115efbc9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:46.662719 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-869b157a-9f4a-419b-a6a8-262115efbc9b None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:46.663928 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-869b157a-9f4a-419b-a6a8-262115efbc9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:46.686916 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:46.742578 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 499/1022] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:46 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:55:46.767975 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060795f3-6839-4c56-bf5a-3d0bf789f416 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:46.901870 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060795f3-6839-4c56-bf5a-3d0bf789f416 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:46.937273 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8072046d-2d3b-4f02-bbe1-356a2564f261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:46.938077 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072046d-2d3b-4f02-bbe1-356a2564f261 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:46.938364 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072046d-2d3b-4f02-bbe1-356a2564f261 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:46.938641 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8072046d-2d3b-4f02-bbe1-356a2564f261 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:46.939856 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8072046d-2d3b-4f02-bbe1-356a2564f261 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:46.941298 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8072046d-2d3b-4f02-bbe1-356a2564f261 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:46.943073 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 524/1023] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:46.948678 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869b157a-9f4a-419b-a6a8-262115efbc9b None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:47.047934 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869b157a-9f4a-419b-a6a8-262115efbc9b None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:47.081171 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd8022-1f35-4138-a5d7-5cf6ed596134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:47.081845 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd8022-1f35-4138-a5d7-5cf6ed596134 None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:47.082135 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd8022-1f35-4138-a5d7-5cf6ed596134 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:47.082418 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd8022-1f35-4138-a5d7-5cf6ed596134 None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:47.083123 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd8022-1f35-4138-a5d7-5cf6ed596134 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:47.085130 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedd8022-1f35-4138-a5d7-5cf6ed596134 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:47.177568 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 500/1024] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 6 16:55:46 2022] POST /identity/v3/users => generated 285 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 16:55:47.351496 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8a3ae-42d9-48da-a860-11801294da48 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:47.351902 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8a3ae-42d9-48da-a860-11801294da48 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:47.352108 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8a3ae-42d9-48da-a860-11801294da48 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:47.353501 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 525/1025] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:47.358822 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a642d8a-1ecb-4685-b230-baaf540e3f56 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:47.359112 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a642d8a-1ecb-4685-b230-baaf540e3f56 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:47.359389 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a642d8a-1ecb-4685-b230-baaf540e3f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:47.377931 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-7a642d8a-1ecb-4685-b230-baaf540e3f56 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:47.379123 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-7a642d8a-1ecb-4685-b230-baaf540e3f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:47.382018 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:47.449129 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 501/1026] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:47 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:55:47.471565 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8a3ae-42d9-48da-a860-11801294da48 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:47.534191 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:47.576546 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8a3ae-42d9-48da-a860-11801294da48 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:47.609595 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72925e00-6be3-4f94-8035-e211445902b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:47.610407 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72925e00-6be3-4f94-8035-e211445902b9 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:47.610688 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72925e00-6be3-4f94-8035-e211445902b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:47.610959 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72925e00-6be3-4f94-8035-e211445902b9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:47.621467 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72925e00-6be3-4f94-8035-e211445902b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:55:47.629585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-72925e00-6be3-4f94-8035-e211445902b9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 06 16:55:47.629585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:47.629585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:47.629585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:47.629585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:47.629585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:47.629585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:47.629585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:47.629585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:47.629585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:47.629585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:47.629585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:47.629585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:55:47.629585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:55:47.629585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:55:47.629585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:55:47.630589 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:55:47.630589 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:55:47.630589 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:55:47.630589 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:55:47.630589 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:47.630589 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:47.630589 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:47.630589 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:47.630589 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:47.630589 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:47.630589 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:47.630589 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:47.630589 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:47.630589 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:47.630589 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:47.630589 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:47.631433 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:47.631433 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:47.631433 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:47.631433 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:47.631433 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:55:47.631433 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:55:47.631433 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:55:47.631433 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:55:47.631433 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:55:47.631433 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:55:47.631433 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 06 16:55:47.631433 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:47.631433 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 526/1027] 10.222.0.40 () {56 vars in 1120 bytes} [Wed Jul 6 16:55:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:47.638401 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a642d8a-1ecb-4685-b230-baaf540e3f56 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:47.747753 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a642d8a-1ecb-4685-b230-baaf540e3f56 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:47.782052 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b3e5f8-8b7a-42a9-ab1a-cae9a49f5782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:47.782724 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3e5f8-8b7a-42a9-ab1a-cae9a49f5782 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:47.783004 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3e5f8-8b7a-42a9-ab1a-cae9a49f5782 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:47.783277 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b3e5f8-8b7a-42a9-ab1a-cae9a49f5782 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:47.784044 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b3e5f8-8b7a-42a9-ab1a-cae9a49f5782 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:47.784351 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b3e5f8-8b7a-42a9-ab1a-cae9a49f5782 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:47.785707 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b3e5f8-8b7a-42a9-ab1a-cae9a49f5782 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:47.798541 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 502/1028] 10.222.0.40 () {56 vars in 1115 bytes} [Wed Jul 6 16:55:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:47.800286 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72925e00-6be3-4f94-8035-e211445902b9 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:47.810739 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85245c5-b2e8-4809-80dd-f4db9f3764c4 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:47.811027 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85245c5-b2e8-4809-80dd-f4db9f3764c4 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:47.811301 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85245c5-b2e8-4809-80dd-f4db9f3764c4 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:47.812824 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 503/1029] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 16:55:47.817311 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40cf31c-29b2-4a76-9f07-2ec945446727 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:47.817601 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40cf31c-29b2-4a76-9f07-2ec945446727 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:47.817885 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40cf31c-29b2-4a76-9f07-2ec945446727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:47.822810 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:47.835754 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-a40cf31c-29b2-4a76-9f07-2ec945446727 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:47.836886 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-a40cf31c-29b2-4a76-9f07-2ec945446727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:47.902779 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72925e00-6be3-4f94-8035-e211445902b9 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:47.905663 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 504/1030] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:47 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 16:55:47.925700 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:47.928360 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40cf31c-29b2-4a76-9f07-2ec945446727 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:47.938554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81605739-1c4c-4d5a-af18-2ebe0e7e2eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:47.939428 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81605739-1c4c-4d5a-af18-2ebe0e7e2eee None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:47.939720 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81605739-1c4c-4d5a-af18-2ebe0e7e2eee None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:47.940027 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81605739-1c4c-4d5a-af18-2ebe0e7e2eee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:47.950830 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81605739-1c4c-4d5a-af18-2ebe0e7e2eee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:47.952312 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81605739-1c4c-4d5a-af18-2ebe0e7e2eee None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:47.970234 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-81605739-1c4c-4d5a-af18-2ebe0e7e2eee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:47.970234 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:47.970234 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:47.970234 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:47.970234 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:47.970234 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:47.970234 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:47.970234 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:47.970234 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:47.970234 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:47.970234 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:47.970234 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:47.970234 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:55:47.970234 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:55:47.970234 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:55:47.971341 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:55:47.971341 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:47.971341 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:47.971341 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:47.971341 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:47.971341 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:47.971341 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:47.971341 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:47.971341 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:47.971341 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:47.971341 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:47.971341 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:47.971341 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:47.971341 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:47.971341 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:47.971341 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:47.971341 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:47.972280 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:55:47.972280 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:55:47.972280 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:47.972280 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:47.972280 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 527/1031] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:55:47.977395 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81605739-1c4c-4d5a-af18-2ebe0e7e2eee None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:48.040137 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40cf31c-29b2-4a76-9f07-2ec945446727 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:48.070296 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:48.076523 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb08a9-fd9e-4d16-b80e-9d5d61f1bdcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:48.077888 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb08a9-fd9e-4d16-b80e-9d5d61f1bdcf None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:48.078198 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb08a9-fd9e-4d16-b80e-9d5d61f1bdcf None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:48.078443 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb08a9-fd9e-4d16-b80e-9d5d61f1bdcf None admin] PATH_INFO: `/v3/services/image` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:48.079246 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafb08a9-fd9e-4d16-b80e-9d5d61f1bdcf None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:48.081190 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafb08a9-fd9e-4d16-b80e-9d5d61f1bdcf None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:48.084633 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81605739-1c4c-4d5a-af18-2ebe0e7e2eee None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:48.093140 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-eafb08a9-fd9e-4d16-b80e-9d5d61f1bdcf None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 06 16:55:48.093140 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:48.093140 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 16:55:48.093140 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 16:55:48.093140 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 16:55:48.093140 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 16:55:48.093140 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 16:55:48.093140 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:48.093140 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 06 16:55:48.093140 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:48.093140 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 16:55:48.093140 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:48.093140 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:48.093140 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:48.093140 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:48.093140 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:48.093140 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:48.094178 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:48.094178 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:48.094178 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:48.094178 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:48.094178 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:48.094178 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:48.094178 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 16:55:48.094178 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 16:55:48.094178 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 16:55:48.094178 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 16:55:48.094178 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:48.094178 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:48.094178 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:48.094178 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:48.094178 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:48.094178 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:48.095028 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:48.095028 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:48.095028 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:48.095028 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:48.095028 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:48.095028 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:48.095028 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:48.095028 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:48.095028 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:48.095028 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:48.095028 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 16:55:48.095028 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:48.095028 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 06 16:55:48.095028 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:48.095028 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 505/1032] 10.222.0.40 () {56 vars in 1105 bytes} [Wed Jul 6 16:55:47 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:55:48.100857 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb08a9-fd9e-4d16-b80e-9d5d61f1bdcf None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:48.120839 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc3852-c544-4687-988a-1925146a88b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:48.121505 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc3852-c544-4687-988a-1925146a88b0 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:48.121790 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc3852-c544-4687-988a-1925146a88b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:48.122065 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edc3852-c544-4687-988a-1925146a88b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:48.122784 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edc3852-c544-4687-988a-1925146a88b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:48.123092 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edc3852-c544-4687-988a-1925146a88b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:48.124473 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edc3852-c544-4687-988a-1925146a88b0 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:48.140428 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 528/1033] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:55:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:48.145502 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc3852-c544-4687-988a-1925146a88b0 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:48.180834 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:48.206074 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb08a9-fd9e-4d16-b80e-9d5d61f1bdcf None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:48.241549 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333fa263-afe2-4632-a90a-74e180472533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:48.242229 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333fa263-afe2-4632-a90a-74e180472533 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:48.242517 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333fa263-afe2-4632-a90a-74e180472533 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:48.242797 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333fa263-afe2-4632-a90a-74e180472533 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:48.243789 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333fa263-afe2-4632-a90a-74e180472533 None admin] RBAC: Adding query filter params (name=image) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:48.244106 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333fa263-afe2-4632-a90a-74e180472533 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:48.245482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333fa263-afe2-4632-a90a-74e180472533 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:48.246559 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edc3852-c544-4687-988a-1925146a88b0 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:48.255361 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 506/1034] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:48 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:48.256971 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333fa263-afe2-4632-a90a-74e180472533 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:48.285934 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54b75a6-e792-419f-b4a7-f1f043fdb3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:48.289510 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b75a6-e792-419f-b4a7-f1f043fdb3d4 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:48.289510 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b75a6-e792-419f-b4a7-f1f043fdb3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:48.289510 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54b75a6-e792-419f-b4a7-f1f043fdb3d4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:48.312725 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54b75a6-e792-419f-b4a7-f1f043fdb3d4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:48.312725 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54b75a6-e792-419f-b4a7-f1f043fdb3d4 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:48.312725 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-f54b75a6-e792-419f-b4a7-f1f043fdb3d4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 06 16:55:48.312725 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:48.312725 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:48.312725 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:48.312725 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:48.312725 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:48.312725 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:48.312725 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:48.312725 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:48.312725 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:48.313933 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:48.313933 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:48.313933 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 16:55:48.313933 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 16:55:48.313933 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 16:55:48.313933 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 16:55:48.313933 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:48.313933 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:48.313933 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 16:55:48.313933 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:48.313933 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 16:55:48.313933 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:48.313933 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:48.313933 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:48.313933 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:48.313933 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:48.313933 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:48.313933 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:48.315286 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:48.315286 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:48.315286 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:48.315286 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:48.315286 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:48.315286 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:48.315286 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:48.315286 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:48.315286 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 16:55:48.315286 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 16:55:48.315286 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 16:55:48.315286 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 16:55:48.315286 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 16:55:48.315286 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 16:55:48.315286 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 06 16:55:48.315286 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:48.315286 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 529/1035] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Jul 6 16:55:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:55:48.319348 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54b75a6-e792-419f-b4a7-f1f043fdb3d4 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:48.327050 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:48.371607 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333fa263-afe2-4632-a90a-74e180472533 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:48.406495 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ba8b2-d1ea-46d2-ac4c-c8073400da55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:48.407163 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ba8b2-d1ea-46d2-ac4c-c8073400da55 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:48.407446 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ba8b2-d1ea-46d2-ac4c-c8073400da55 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:48.407720 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ba8b2-d1ea-46d2-ac4c-c8073400da55 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:48.408458 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ba8b2-d1ea-46d2-ac4c-c8073400da55 None admin] RBAC: Adding query filter params (type=image) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:48.408808 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ba8b2-d1ea-46d2-ac4c-c8073400da55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:48.410173 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0ba8b2-d1ea-46d2-ac4c-c8073400da55 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:48.419795 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 507/1036] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:48 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:48.420894 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ba8b2-d1ea-46d2-ac4c-c8073400da55 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:48.426447 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:48.432985 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54b75a6-e792-419f-b4a7-f1f043fdb3d4 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:48.468938 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bb13a3-c457-48f1-932e-e0207f3c63dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:48.469629 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb13a3-c457-48f1-932e-e0207f3c63dd None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:48.469892 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb13a3-c457-48f1-932e-e0207f3c63dd None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:48.470165 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bb13a3-c457-48f1-932e-e0207f3c63dd None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:48.470998 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bb13a3-c457-48f1-932e-e0207f3c63dd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:48.471297 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bb13a3-c457-48f1-932e-e0207f3c63dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:48.472755 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bb13a3-c457-48f1-932e-e0207f3c63dd None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:48.490901 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 530/1037] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Jul 6 16:55:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:48.496409 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bb13a3-c457-48f1-932e-e0207f3c63dd None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:48.530721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ba8b2-d1ea-46d2-ac4c-c8073400da55 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:48.557278 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:48.570145 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db329b2-d37f-43f6-bfb1-205867416bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:48.570145 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db329b2-d37f-43f6-bfb1-205867416bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:48.570145 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db329b2-d37f-43f6-bfb1-205867416bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:48.570428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db329b2-d37f-43f6-bfb1-205867416bc3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:48.571150 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db329b2-d37f-43f6-bfb1-205867416bc3 None admin] RBAC: Adding query filter params (interface=public, service_id=0ca2aca80eed426b89b1022798eae62b, region_id=RegionOne) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:48.571468 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db329b2-d37f-43f6-bfb1-205867416bc3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:48.573250 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db329b2-d37f-43f6-bfb1-205867416bc3 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:48.580089 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 508/1038] 10.222.0.40 () {56 vars in 1251 bytes} [Wed Jul 6 16:55:48 2022] GET /identity/v3/endpoints?service_id=0ca2aca80eed426b89b1022798eae62b&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:48.586204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db329b2-d37f-43f6-bfb1-205867416bc3 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:48.601761 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bb13a3-c457-48f1-932e-e0207f3c63dd None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:48.635158 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb7e02d-1e30-4aa3-94d8-496c927d8485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:48.635845 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb7e02d-1e30-4aa3-94d8-496c927d8485 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:48.636131 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb7e02d-1e30-4aa3-94d8-496c927d8485 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:48.636405 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb7e02d-1e30-4aa3-94d8-496c927d8485 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:48.657815 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:48.697829 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db329b2-d37f-43f6-bfb1-205867416bc3 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:48.704723 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb7e02d-1e30-4aa3-94d8-496c927d8485 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:48.706181 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb7e02d-1e30-4aa3-94d8-496c927d8485 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:48.742622 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff55a80-c783-4b87-98ed-457b4d7bb578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:48.743356 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff55a80-c783-4b87-98ed-457b4d7bb578 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:48.743770 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff55a80-c783-4b87-98ed-457b4d7bb578 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:48.743969 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff55a80-c783-4b87-98ed-457b4d7bb578 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:48.744660 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff55a80-c783-4b87-98ed-457b4d7bb578 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:48.746000 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff55a80-c783-4b87-98ed-457b4d7bb578 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:48.763755 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 509/1039] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Jul 6 16:55:48 2022] GET /identity/v3/services => generated 1957 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 16:55:48.832318 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 531/1040] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:55:48 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 06 16:55:48.838834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff55a80-c783-4b87-98ed-457b4d7bb578 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:48.897567 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:48.948864 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff55a80-c783-4b87-98ed-457b4d7bb578 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:48.983352 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79b80a-109f-4b42-9ebd-5616bc091419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:48.984179 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79b80a-109f-4b42-9ebd-5616bc091419 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:48.984465 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79b80a-109f-4b42-9ebd-5616bc091419 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:48.984740 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79b80a-109f-4b42-9ebd-5616bc091419 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:48.995408 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b79b80a-109f-4b42-9ebd-5616bc091419 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:48.996878 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b79b80a-109f-4b42-9ebd-5616bc091419 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:49.014428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-5b79b80a-109f-4b42-9ebd-5616bc091419 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:49.014428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:49.014428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:49.014428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:49.014428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:49.014428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:49.014428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:49.014428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:49.014428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:49.014428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:49.014428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:49.014428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:49.014428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:55:49.014428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:55:49.014428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:55:49.015325 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:55:49.015325 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:49.015325 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:49.015325 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:49.015325 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:49.015325 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:49.015325 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:49.015325 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:49.015325 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:49.015325 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:49.015325 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:49.015325 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:49.015325 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:49.015325 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:49.015325 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:49.015325 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:49.015325 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:49.016254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:55:49.016254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:55:49.016254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:49.016254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:49.016254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 510/1041] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:49.022772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb7e02d-1e30-4aa3-94d8-496c927d8485 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:49.077654 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:49.124902 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb7e02d-1e30-4aa3-94d8-496c927d8485 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:49.158956 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc380b3-972f-4eec-9e19-80d903892c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:49.159801 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc380b3-972f-4eec-9e19-80d903892c7e None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:49.160084 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc380b3-972f-4eec-9e19-80d903892c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:49.160357 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc380b3-972f-4eec-9e19-80d903892c7e None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:49.161080 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc380b3-972f-4eec-9e19-80d903892c7e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:49.161385 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc380b3-972f-4eec-9e19-80d903892c7e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:49.162768 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc380b3-972f-4eec-9e19-80d903892c7e None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:49.179045 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 532/1042] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:55:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:49.180705 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79b80a-109f-4b42-9ebd-5616bc091419 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:49.266871 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933594d1-15cc-4ca4-ba3e-ebb24e1ace6c None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:49.267218 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933594d1-15cc-4ca4-ba3e-ebb24e1ace6c None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:49.267441 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933594d1-15cc-4ca4-ba3e-ebb24e1ace6c None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:49.268845 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 533/1043] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 16:55:49.274064 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b04f9b-7b56-4734-b442-7e7f3a8b90c6 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:49.274336 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b04f9b-7b56-4734-b442-7e7f3a8b90c6 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:49.274618 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b04f9b-7b56-4734-b442-7e7f3a8b90c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:49.288553 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79b80a-109f-4b42-9ebd-5616bc091419 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:49.290933 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:49.293456 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-07b04f9b-7b56-4734-b442-7e7f3a8b90c6 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:49.294644 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-07b04f9b-7b56-4734-b442-7e7f3a8b90c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:49.327226 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3a9417-b7bf-465c-ac42-83a3c9694b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:49.328058 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a9417-b7bf-465c-ac42-83a3c9694b49 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:49.328344 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a9417-b7bf-465c-ac42-83a3c9694b49 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:49.328618 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3a9417-b7bf-465c-ac42-83a3c9694b49 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:49.335291 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3a9417-b7bf-465c-ac42-83a3c9694b49 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:49.336769 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3a9417-b7bf-465c-ac42-83a3c9694b49 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:49.350501 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-ee3a9417-b7bf-465c-ac42-83a3c9694b49 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:49.350501 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:49.350501 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:49.350501 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:49.350501 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:49.350501 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:49.350501 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:49.350501 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:49.350501 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:49.350501 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:49.350501 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:49.350501 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:49.350501 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 16:55:49.350501 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 16:55:49.350501 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 16:55:49.351428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 16:55:49.351428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:49.351428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:49.351428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:49.351428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:49.351428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:49.351428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:49.351428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:49.351428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:49.351428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:49.351428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:49.351428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:49.351428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:49.351428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:49.351428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:49.351428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:49.351428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:49.352345 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 16:55:49.352345 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 16:55:49.352345 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 16:55:49.352345 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 16:55:49.352345 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 16:55:49.352345 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 16:55:49.352345 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:49.352345 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:49.352345 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 511/1044] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:55:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:49.357400 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3a9417-b7bf-465c-ac42-83a3c9694b49 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:49.371763 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 534/1045] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:49 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 16:55:49.393573 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b04f9b-7b56-4734-b442-7e7f3a8b90c6 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:49.438423 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:49.464981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3a9417-b7bf-465c-ac42-83a3c9694b49 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:49.497333 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b04f9b-7b56-4734-b442-7e7f3a8b90c6 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:49.499628 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b7f818-0091-4813-b1c7-95c6a4a58150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:49.500326 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7f818-0091-4813-b1c7-95c6a4a58150 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:49.500613 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7f818-0091-4813-b1c7-95c6a4a58150 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:49.500890 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b7f818-0091-4813-b1c7-95c6a4a58150 None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:49.501677 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b7f818-0091-4813-b1c7-95c6a4a58150 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:49.501987 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b7f818-0091-4813-b1c7-95c6a4a58150 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:49.503418 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b7f818-0091-4813-b1c7-95c6a4a58150 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:49.521908 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 512/1046] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 6 16:55:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:49.523953 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b7f818-0091-4813-b1c7-95c6a4a58150 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:49.536506 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c866a36-09ff-4ede-846e-73e5431d518e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:49.537417 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c866a36-09ff-4ede-846e-73e5431d518e None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:49.537623 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c866a36-09ff-4ede-846e-73e5431d518e None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:49.537940 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c866a36-09ff-4ede-846e-73e5431d518e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:49.548195 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c866a36-09ff-4ede-846e-73e5431d518e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:55:49.550370 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:49.556422 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-3c866a36-09ff-4ede-846e-73e5431d518e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:55:49.556422 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:49.556422 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:49.556422 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:49.556422 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:49.556422 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:49.556422 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:49.556422 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:49.556422 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:49.556422 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:49.556422 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:49.556422 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:49.556422 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:55:49.556422 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:55:49.556422 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:55:49.556422 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:55:49.557562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:55:49.557562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:55:49.557562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:55:49.557562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:55:49.557562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:49.557562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:49.557562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:49.557562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:49.557562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:49.557562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:49.557562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:49.557562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:49.557562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:49.557562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:49.557562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:49.557562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:49.558405 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:49.558405 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:49.558405 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:49.558405 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:49.558405 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:55:49.558405 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:55:49.558405 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:55:49.558405 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:55:49.558405 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:55:49.558405 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:55:49.558405 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:55:49.558405 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:49.558405 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 535/1047] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 6 16:55:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:55:49.563512 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c866a36-09ff-4ede-846e-73e5431d518e None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:49.632731 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b7f818-0091-4813-b1c7-95c6a4a58150 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:49.667573 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6513f3-adbb-4fcd-bd4e-9b9144587197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:49.668273 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6513f3-adbb-4fcd-bd4e-9b9144587197 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:49.668404 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c866a36-09ff-4ede-846e-73e5431d518e None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:49.668586 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6513f3-adbb-4fcd-bd4e-9b9144587197 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:49.668830 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6513f3-adbb-4fcd-bd4e-9b9144587197 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:49.669644 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6513f3-adbb-4fcd-bd4e-9b9144587197 None admin] RBAC: Adding query filter params (role.id=4d3a77c1547c43aba11b60377d04d537, scope.project.id=7938b0346d4240ab89774327d09a89ae, user.id=af26ef96dc9d47dc91549b736939268e) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:49.669958 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6513f3-adbb-4fcd-bd4e-9b9144587197 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:49.670182 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:49.671383 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6513f3-adbb-4fcd-bd4e-9b9144587197 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:49.677269 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 513/1048] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:55:49 2022] GET /identity/v3/role_assignments?user.id=af26ef96dc9d47dc91549b736939268e&scope.project.id=7938b0346d4240ab89774327d09a89ae&role.id=4d3a77c1547c43aba11b60377d04d537 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:49.706241 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e062c01e-1626-409c-9f57-37b8eb9c584f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:49.707328 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062c01e-1626-409c-9f57-37b8eb9c584f None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:49.707851 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062c01e-1626-409c-9f57-37b8eb9c584f None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:49.708368 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e062c01e-1626-409c-9f57-37b8eb9c584f None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:49.709361 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e062c01e-1626-409c-9f57-37b8eb9c584f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:49.709907 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e062c01e-1626-409c-9f57-37b8eb9c584f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:49.711567 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e062c01e-1626-409c-9f57-37b8eb9c584f None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:49.721693 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 536/1049] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:55:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:49.727612 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6513f3-adbb-4fcd-bd4e-9b9144587197 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:49.824835 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:49.840700 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6513f3-adbb-4fcd-bd4e-9b9144587197 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:49.878755 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e816a-b61c-4bd6-919a-e9df11c96b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:49.879652 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e816a-b61c-4bd6-919a-e9df11c96b74 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:49.879972 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e816a-b61c-4bd6-919a-e9df11c96b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:49.880422 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37e816a-b61c-4bd6-919a-e9df11c96b74 None admin] PATH_INFO: `/v3/users/a8cf1915bcb74d419cdb783bbcde8e0e` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:49.894814 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37e816a-b61c-4bd6-919a-e9df11c96b74 None admin] RBAC: Authorizing `identity:get_user(user_id=a8cf1915bcb74d419cdb783bbcde8e0e)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:49.897197 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37e816a-b61c-4bd6-919a-e9df11c96b74 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:49.903464 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 514/1050] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:55:49 2022] GET /identity/v3/users/a8cf1915bcb74d419cdb783bbcde8e0e => generated 285 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:49.904873 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e062c01e-1626-409c-9f57-37b8eb9c584f None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:50.011106 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e062c01e-1626-409c-9f57-37b8eb9c584f None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:50.044380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2be7a72-d693-4e49-884d-818507588550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:50.045273 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be7a72-d693-4e49-884d-818507588550 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:50.045705 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be7a72-d693-4e49-884d-818507588550 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:50.046126 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2be7a72-d693-4e49-884d-818507588550 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:50.076427 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:50.113986 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2be7a72-d693-4e49-884d-818507588550 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:50.115463 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2be7a72-d693-4e49-884d-818507588550 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:50.219273 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 537/1051] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:55:49 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 06 16:55:50.219474 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e816a-b61c-4bd6-919a-e9df11c96b74 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:50.322075 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37e816a-b61c-4bd6-919a-e9df11c96b74 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:50.356143 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b2610-6f1d-49bd-9427-fe28999b10ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:50.357011 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b2610-6f1d-49bd-9427-fe28999b10ca None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:50.357299 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b2610-6f1d-49bd-9427-fe28999b10ca None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:50.357580 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b2610-6f1d-49bd-9427-fe28999b10ca None admin] PATH_INFO: `/v3/projects/a0c8d8821b4b41888b7ea09ee950be20` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:50.358759 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b2610-6f1d-49bd-9427-fe28999b10ca None admin] RBAC: Authorizing `identity:get_project(project_id=a0c8d8821b4b41888b7ea09ee950be20)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:50.360222 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b2610-6f1d-49bd-9427-fe28999b10ca None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:50.362097 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 515/1052] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:55:50 2022] GET /identity/v3/projects/a0c8d8821b4b41888b7ea09ee950be20 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:50.367924 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2be7a72-d693-4e49-884d-818507588550 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:50.430307 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:50.468856 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2be7a72-d693-4e49-884d-818507588550 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:50.501606 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3658b6f9-9431-4ec9-b8cd-96c69691b505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:50.502260 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658b6f9-9431-4ec9-b8cd-96c69691b505 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:50.502546 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658b6f9-9431-4ec9-b8cd-96c69691b505 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:50.502821 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3658b6f9-9431-4ec9-b8cd-96c69691b505 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:50.503797 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3658b6f9-9431-4ec9-b8cd-96c69691b505 None admin] RBAC: Adding query filter params (role.id=b41f8a7a58e64f749f2099ca45cc04a7, scope.project.id=a0c8d8821b4b41888b7ea09ee950be20, user.id=a8cf1915bcb74d419cdb783bbcde8e0e) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:50.504115 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3658b6f9-9431-4ec9-b8cd-96c69691b505 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:50.505589 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3658b6f9-9431-4ec9-b8cd-96c69691b505 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:50.511674 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 538/1053] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:55:50 2022] GET /identity/v3/role_assignments?user.id=a8cf1915bcb74d419cdb783bbcde8e0e&scope.project.id=a0c8d8821b4b41888b7ea09ee950be20&role.id=b41f8a7a58e64f749f2099ca45cc04a7 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:50.786279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa48a9-7bcc-4fd6-968c-3b8b0700b5f8 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:50.786592 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa48a9-7bcc-4fd6-968c-3b8b0700b5f8 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:50.786837 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fa48a9-7bcc-4fd6-968c-3b8b0700b5f8 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:50.789020 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 516/1054] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:50.793913 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919d19b-f9f0-476f-888a-bd9a8fa4dfc4 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:50.794209 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919d19b-f9f0-476f-888a-bd9a8fa4dfc4 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:50.794490 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919d19b-f9f0-476f-888a-bd9a8fa4dfc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:50.811284 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-a919d19b-f9f0-476f-888a-bd9a8fa4dfc4 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:50.812488 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-a919d19b-f9f0-476f-888a-bd9a8fa4dfc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:50.883714 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 539/1055] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:50 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:55:50.902336 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fa48a9-7bcc-4fd6-968c-3b8b0700b5f8 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:50.918744 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:51.003856 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fa48a9-7bcc-4fd6-968c-3b8b0700b5f8 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:51.037192 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a23be8-2f45-4b47-b768-480aa3eed169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:51.037976 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a23be8-2f45-4b47-b768-480aa3eed169 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:51.038258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a23be8-2f45-4b47-b768-480aa3eed169 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:51.038530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a23be8-2f45-4b47-b768-480aa3eed169 None admin] PATH_INFO: `/v3/services/image` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:51.039172 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a23be8-2f45-4b47-b768-480aa3eed169 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:51.040554 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a23be8-2f45-4b47-b768-480aa3eed169 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:51.052152 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-f3a23be8-2f45-4b47-b768-480aa3eed169 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 06 16:55:51.052152 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:51.052152 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 16:55:51.052152 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 16:55:51.052152 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 16:55:51.052152 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 16:55:51.052152 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 16:55:51.052152 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:51.052152 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 06 16:55:51.052152 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:51.052152 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 16:55:51.052152 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:51.052152 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:51.052152 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:51.052152 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:51.052152 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:51.052152 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:51.053637 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:51.053637 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:51.053637 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:51.053637 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:51.053637 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:51.053637 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:51.053637 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 16:55:51.053637 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 16:55:51.053637 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 16:55:51.053637 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 16:55:51.053637 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:51.053637 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:51.053637 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:51.053637 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:51.053637 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:51.053637 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:51.054514 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:51.054514 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:51.054514 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:51.054514 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:51.054514 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:51.054514 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:51.054514 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:51.054514 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:51.054514 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:51.054514 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:51.054514 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 16:55:51.054514 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:55:51.054514 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 06 16:55:51.054514 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:51.054514 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 517/1056] 10.222.0.40 () {56 vars in 1105 bytes} [Wed Jul 6 16:55:50 2022] GET /identity/v3/services/image => generated 86 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:51.061416 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a919d19b-f9f0-476f-888a-bd9a8fa4dfc4 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:51.119207 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:51.160891 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a919d19b-f9f0-476f-888a-bd9a8fa4dfc4 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:51.194990 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c7a0d-6c26-4521-ab0c-26d16c36bfcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:51.195664 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c7a0d-6c26-4521-ab0c-26d16c36bfcc None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:51.195977 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c7a0d-6c26-4521-ab0c-26d16c36bfcc None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:51.196253 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0c7a0d-6c26-4521-ab0c-26d16c36bfcc None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:51.196956 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0c7a0d-6c26-4521-ab0c-26d16c36bfcc None admin] RBAC: Adding query filter params (name=image) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:51.197258 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0c7a0d-6c26-4521-ab0c-26d16c36bfcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:51.198614 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0c7a0d-6c26-4521-ab0c-26d16c36bfcc None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:51.208378 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 540/1057] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:51 2022] GET /identity/v3/services?name=image => generated 122 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:51.209352 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a23be8-2f45-4b47-b768-480aa3eed169 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:51.289091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:51.310013 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a23be8-2f45-4b47-b768-480aa3eed169 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:51.344323 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc396839-ccd6-48ee-8cf1-ee73c7d3efa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:51.344981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc396839-ccd6-48ee-8cf1-ee73c7d3efa5 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:51.345262 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc396839-ccd6-48ee-8cf1-ee73c7d3efa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:51.345533 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc396839-ccd6-48ee-8cf1-ee73c7d3efa5 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:51.346233 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc396839-ccd6-48ee-8cf1-ee73c7d3efa5 None admin] RBAC: Adding query filter params (type=image) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:51.346535 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc396839-ccd6-48ee-8cf1-ee73c7d3efa5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:51.348007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc396839-ccd6-48ee-8cf1-ee73c7d3efa5 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:51.359091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 518/1058] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:51 2022] GET /identity/v3/services?type=image => generated 350 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:51.359255 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c7a0d-6c26-4521-ab0c-26d16c36bfcc None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:51.460863 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0c7a0d-6c26-4521-ab0c-26d16c36bfcc None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:51.483155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:51.495296 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feebb51-21b3-4351-a395-2ad7b4b8d72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:51.496391 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feebb51-21b3-4351-a395-2ad7b4b8d72b None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:51.496391 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feebb51-21b3-4351-a395-2ad7b4b8d72b None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:51.496554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feebb51-21b3-4351-a395-2ad7b4b8d72b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:51.497254 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feebb51-21b3-4351-a395-2ad7b4b8d72b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:51.498617 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feebb51-21b3-4351-a395-2ad7b4b8d72b None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:51.579840 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 541/1059] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 6 16:55:51 2022] POST /identity/v3/endpoints => generated 327 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 16:55:51.697442 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6435143b-21ab-4e01-8d6c-66eb0ed172b7 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:51.697734 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6435143b-21ab-4e01-8d6c-66eb0ed172b7 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:51.698014 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6435143b-21ab-4e01-8d6c-66eb0ed172b7 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:51.699407 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 519/1060] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 16:55:51.705186 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ad651-1b6e-4fe4-ac98-ac9ead90f85c None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:51.705550 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ad651-1b6e-4fe4-ac98-ac9ead90f85c None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:51.705899 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82ad651-1b6e-4fe4-ac98-ac9ead90f85c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:51.723951 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-f82ad651-1b6e-4fe4-ac98-ac9ead90f85c None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:51.725234 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f82ad651-1b6e-4fe4-ac98-ac9ead90f85c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:51.801583 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 542/1061] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 16:55:51.824105 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6435143b-21ab-4e01-8d6c-66eb0ed172b7 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:51.926302 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6435143b-21ab-4e01-8d6c-66eb0ed172b7 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:51.960108 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c9ec63-77cf-48d6-90bb-cc1f701975d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:51.960925 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9ec63-77cf-48d6-90bb-cc1f701975d9 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:51.961196 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9ec63-77cf-48d6-90bb-cc1f701975d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:51.961470 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9ec63-77cf-48d6-90bb-cc1f701975d9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:51.971473 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c9ec63-77cf-48d6-90bb-cc1f701975d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:55:51.980198 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-68c9ec63-77cf-48d6-90bb-cc1f701975d9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 06 16:55:51.980198 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:51.980198 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:51.980198 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:51.980198 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:51.980198 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:51.980198 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:51.980198 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:51.980198 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:51.980198 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:51.980198 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:51.980198 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:51.980198 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:55:51.980198 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:55:51.980198 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:55:51.980198 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:55:51.981194 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:55:51.981194 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:55:51.981194 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:55:51.981194 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:55:51.981194 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:51.981194 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:51.981194 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:51.981194 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:51.981194 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:51.981194 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:51.981194 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:51.981194 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:51.981194 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:51.981194 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:51.981194 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:51.981194 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:51.987105 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:51.987105 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:51.987105 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:51.987105 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:51.987105 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:55:51.987105 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:55:51.987105 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:55:51.987105 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:55:51.987105 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:55:51.987105 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:55:51.987105 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 06 16:55:51.987105 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:51.987105 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 520/1062] 10.222.0.40 () {56 vars in 1120 bytes} [Wed Jul 6 16:55:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:55:51.989433 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ad651-1b6e-4fe4-ac98-ac9ead90f85c None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:52.093210 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82ad651-1b6e-4fe4-ac98-ac9ead90f85c None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:52.095565 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:52.127990 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5157f1a5-3dbe-42fc-b706-44cd942f9e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:52.128672 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5157f1a5-3dbe-42fc-b706-44cd942f9e19 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:52.128957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5157f1a5-3dbe-42fc-b706-44cd942f9e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:52.129234 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5157f1a5-3dbe-42fc-b706-44cd942f9e19 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:52.129979 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5157f1a5-3dbe-42fc-b706-44cd942f9e19 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:52.130286 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5157f1a5-3dbe-42fc-b706-44cd942f9e19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:52.131686 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5157f1a5-3dbe-42fc-b706-44cd942f9e19 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:52.140016 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 543/1063] 10.222.0.40 () {56 vars in 1115 bytes} [Wed Jul 6 16:55:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:52.145649 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c9ec63-77cf-48d6-90bb-cc1f701975d9 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:52.149341 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:52.248243 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c9ec63-77cf-48d6-90bb-cc1f701975d9 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:52.282559 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bac77a-d725-4e83-a5d0-5c11c14e27f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:52.283415 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bac77a-d725-4e83-a5d0-5c11c14e27f0 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:52.283694 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bac77a-d725-4e83-a5d0-5c11c14e27f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:52.283995 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bac77a-d725-4e83-a5d0-5c11c14e27f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:52.295131 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bac77a-d725-4e83-a5d0-5c11c14e27f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:52.296620 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bac77a-d725-4e83-a5d0-5c11c14e27f0 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:52.314240 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-a9bac77a-d725-4e83-a5d0-5c11c14e27f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:52.314240 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:52.314240 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:52.314240 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:52.314240 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:52.314240 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:52.314240 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:52.314240 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:52.314240 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:52.314240 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:52.314240 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:52.314240 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:52.314240 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:55:52.314240 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:55:52.314240 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:55:52.315156 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:55:52.315156 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:52.315156 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:52.315156 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:52.315156 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:52.315156 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:52.315156 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:52.315156 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:52.315156 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:52.315156 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:52.315156 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:52.315156 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:52.315156 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:52.315156 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:52.315156 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:52.315156 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:52.315156 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:52.321168 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:55:52.321168 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:55:52.321168 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:52.321168 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:52.321168 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 521/1064] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:55:52.322928 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5157f1a5-3dbe-42fc-b706-44cd942f9e19 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:52.425837 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5157f1a5-3dbe-42fc-b706-44cd942f9e19 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:52.459471 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8287206c-b6c2-43ea-8691-e2901d8a8c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:52.460168 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287206c-b6c2-43ea-8691-e2901d8a8c9d None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:52.460449 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287206c-b6c2-43ea-8691-e2901d8a8c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:52.460721 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8287206c-b6c2-43ea-8691-e2901d8a8c9d None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:52.461435 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8287206c-b6c2-43ea-8691-e2901d8a8c9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:52.461739 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8287206c-b6c2-43ea-8691-e2901d8a8c9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:52.463147 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8287206c-b6c2-43ea-8691-e2901d8a8c9d None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:52.475699 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 544/1065] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:55:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:52.481121 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bac77a-d725-4e83-a5d0-5c11c14e27f0 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:52.533480 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b0b33-72eb-41ce-bdb3-0e9aa469a5d0 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:52.533829 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b0b33-72eb-41ce-bdb3-0e9aa469a5d0 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:52.533974 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b0b33-72eb-41ce-bdb3-0e9aa469a5d0 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:52.535331 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 545/1066] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 16:55:52.537746 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:52.540881 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6be53-129d-4aec-b67d-3b331f9f2ff7 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:52.541173 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6be53-129d-4aec-b67d-3b331f9f2ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:52.541462 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6be53-129d-4aec-b67d-3b331f9f2ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:52.558134 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-abc6be53-129d-4aec-b67d-3b331f9f2ff7 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:52.559321 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-abc6be53-129d-4aec-b67d-3b331f9f2ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:52.580406 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:52.584329 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bac77a-d725-4e83-a5d0-5c11c14e27f0 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:52.619658 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c1a20d-3e8c-4d90-9675-846af7651b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:52.620627 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1a20d-3e8c-4d90-9675-846af7651b75 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:52.621042 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1a20d-3e8c-4d90-9675-846af7651b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:52.621319 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c1a20d-3e8c-4d90-9675-846af7651b75 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:52.631443 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c1a20d-3e8c-4d90-9675-846af7651b75 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:52.632909 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c1a20d-3e8c-4d90-9675-846af7651b75 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:52.633576 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 546/1067] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 16:55:52.650430 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-d0c1a20d-3e8c-4d90-9675-846af7651b75 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 06 16:55:52.650430 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:52.650430 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:52.650430 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:52.650430 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:52.650430 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:52.650430 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:52.650430 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:52.650430 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:52.650430 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:52.650430 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:52.650430 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:52.650430 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 16:55:52.650430 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 16:55:52.650430 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 16:55:52.650430 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 16:55:52.651410 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:52.651410 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:52.651410 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 16:55:52.651410 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:52.651410 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 16:55:52.651410 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:52.651410 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:52.651410 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:52.651410 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:52.651410 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:52.651410 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:52.651410 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:52.651410 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:52.651410 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:52.651410 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:52.651410 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:52.651410 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:52.651410 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:52.652416 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:52.652416 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:52.652416 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 16:55:52.652416 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 16:55:52.652416 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 16:55:52.652416 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 16:55:52.652416 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 16:55:52.652416 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 16:55:52.652416 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 06 16:55:52.652416 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:52.652416 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 522/1068] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Jul 6 16:55:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:52.657829 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6be53-129d-4aec-b67d-3b331f9f2ff7 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:52.658423 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c1a20d-3e8c-4d90-9675-846af7651b75 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:52.768320 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c1a20d-3e8c-4d90-9675-846af7651b75 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:52.773539 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6be53-129d-4aec-b67d-3b331f9f2ff7 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:52.783170 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:52.804042 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee4996-f99b-40a1-a3a2-7ab2ec18b2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:52.804679 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee4996-f99b-40a1-a3a2-7ab2ec18b2f8 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:52.804966 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee4996-f99b-40a1-a3a2-7ab2ec18b2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:52.805236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee4996-f99b-40a1-a3a2-7ab2ec18b2f8 None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:52.806075 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ee4996-f99b-40a1-a3a2-7ab2ec18b2f8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:52.806389 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ee4996-f99b-40a1-a3a2-7ab2ec18b2f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:52.807856 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ee4996-f99b-40a1-a3a2-7ab2ec18b2f8 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:52.815805 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab42e30-43ab-4264-afb4-8adf9daa0ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:52.816488 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab42e30-43ab-4264-afb4-8adf9daa0ade None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:52.816773 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab42e30-43ab-4264-afb4-8adf9daa0ade None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:52.817051 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab42e30-43ab-4264-afb4-8adf9daa0ade None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:52.827152 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 523/1069] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Jul 6 16:55:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:52.827399 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab42e30-43ab-4264-afb4-8adf9daa0ade None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:55:52.832308 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee4996-f99b-40a1-a3a2-7ab2ec18b2f8 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:52.835470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-8ab42e30-43ab-4264-afb4-8adf9daa0ade None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:55:52.835470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:52.835470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:52.835470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:52.835470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:52.835470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:52.835470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:52.835470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:52.835470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:52.835470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:52.835470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:52.835470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:52.835470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:55:52.835470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:55:52.835470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:55:52.835470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:55:52.836411 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:55:52.836411 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:55:52.836411 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:55:52.836411 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:55:52.836411 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:52.836411 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:52.836411 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:52.836411 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:52.836411 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:52.836411 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:52.836411 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:52.836411 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:52.836411 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:52.836411 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:52.836411 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:52.836411 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:52.837257 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:52.837257 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:52.837257 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:52.837257 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:52.837257 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:55:52.837257 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:55:52.837257 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:55:52.837257 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:55:52.837257 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:55:52.837257 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:55:52.837257 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:55:52.837257 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:55:52.837257 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 547/1070] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 6 16:55:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:55:52.842465 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab42e30-43ab-4264-afb4-8adf9daa0ade None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:52.862591 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:52.938195 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee4996-f99b-40a1-a3a2-7ab2ec18b2f8 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:52.955159 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab42e30-43ab-4264-afb4-8adf9daa0ade None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:52.973485 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc7002-deb9-4fcb-a60a-a05aa4eeeb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:52.974278 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc7002-deb9-4fcb-a60a-a05aa4eeeb31 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:52.975092 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc7002-deb9-4fcb-a60a-a05aa4eeeb31 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:52.975416 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc7002-deb9-4fcb-a60a-a05aa4eeeb31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:52.985397 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfc7002-deb9-4fcb-a60a-a05aa4eeeb31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:52.986828 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfc7002-deb9-4fcb-a60a-a05aa4eeeb31 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:52.992586 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3014dd87-051f-4f09-98b8-ac2360ae19dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:52.993255 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3014dd87-051f-4f09-98b8-ac2360ae19dd None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:52.993543 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3014dd87-051f-4f09-98b8-ac2360ae19dd None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:52.993820 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3014dd87-051f-4f09-98b8-ac2360ae19dd None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:52.994958 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3014dd87-051f-4f09-98b8-ac2360ae19dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:52.995272 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3014dd87-051f-4f09-98b8-ac2360ae19dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:52.997033 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3014dd87-051f-4f09-98b8-ac2360ae19dd None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:53.004503 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-8bfc7002-deb9-4fcb-a60a-a05aa4eeeb31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:53.004503 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:53.004503 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:53.004503 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:53.004503 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:53.004503 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:53.004503 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:53.004503 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:53.004503 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:53.004503 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:53.004503 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:53.004503 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:53.004503 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:55:53.004503 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:55:53.004503 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:55:53.010293 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:55:53.010293 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:53.010293 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:53.010293 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:53.010293 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:53.010293 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:53.010293 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:53.010293 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:53.010293 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:53.010293 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:53.010293 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:53.010293 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:53.010293 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:53.010293 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:53.010293 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:53.010293 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:53.010293 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:53.011228 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:55:53.011228 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:55:53.011228 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:53.011228 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:53.011452 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 548/1071] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:55:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:53.011525 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 524/1072] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:55:53.011629 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3014dd87-051f-4f09-98b8-ac2360ae19dd None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:53.011707 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc7002-deb9-4fcb-a60a-a05aa4eeeb31 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:53.037900 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:53.096482 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:53.118121 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3014dd87-051f-4f09-98b8-ac2360ae19dd None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:53.118816 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc7002-deb9-4fcb-a60a-a05aa4eeeb31 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:53.152907 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c9f46-3cec-4f01-bbe9-a59e77462640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:53.154628 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c9f46-3cec-4f01-bbe9-a59e77462640 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:53.154912 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c9f46-3cec-4f01-bbe9-a59e77462640 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:53.155191 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f1c9f46-3cec-4f01-bbe9-a59e77462640 None admin] PATH_INFO: `/v3/users/a8cf1915bcb74d419cdb783bbcde8e0e` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:53.156407 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1c9f46-3cec-4f01-bbe9-a59e77462640 None admin] RBAC: Authorizing `identity:get_user(user_id=a8cf1915bcb74d419cdb783bbcde8e0e)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:53.157443 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae43b6e2-f2c8-4207-8eaa-9f09f1b01e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:53.157833 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f1c9f46-3cec-4f01-bbe9-a59e77462640 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:53.158110 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43b6e2-f2c8-4207-8eaa-9f09f1b01e61 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:53.158398 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43b6e2-f2c8-4207-8eaa-9f09f1b01e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:53.158679 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae43b6e2-f2c8-4207-8eaa-9f09f1b01e61 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:53.159627 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae43b6e2-f2c8-4207-8eaa-9f09f1b01e61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:53.159837 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 549/1073] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:55:53 2022] GET /identity/v3/users/a8cf1915bcb74d419cdb783bbcde8e0e => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:53.159927 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae43b6e2-f2c8-4207-8eaa-9f09f1b01e61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:53.161076 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae43b6e2-f2c8-4207-8eaa-9f09f1b01e61 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:53.165851 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c9f46-3cec-4f01-bbe9-a59e77462640 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:53.178122 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 525/1074] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:55:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:53.179176 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae43b6e2-f2c8-4207-8eaa-9f09f1b01e61 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:53.272266 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:53.274652 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f1c9f46-3cec-4f01-bbe9-a59e77462640 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:53.289237 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae43b6e2-f2c8-4207-8eaa-9f09f1b01e61 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:53.309651 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0fdfae-e6ec-4215-9c97-fc3234410fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:53.310515 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0fdfae-e6ec-4215-9c97-fc3234410fcb None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:53.310801 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0fdfae-e6ec-4215-9c97-fc3234410fcb None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:53.311077 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0fdfae-e6ec-4215-9c97-fc3234410fcb None admin] PATH_INFO: `/v3/projects/a0c8d8821b4b41888b7ea09ee950be20` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:53.312318 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0fdfae-e6ec-4215-9c97-fc3234410fcb None admin] RBAC: Authorizing `identity:get_project(project_id=a0c8d8821b4b41888b7ea09ee950be20)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:53.313749 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0fdfae-e6ec-4215-9c97-fc3234410fcb None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:53.315620 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 550/1075] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:55:53 2022] GET /identity/v3/projects/a0c8d8821b4b41888b7ea09ee950be20 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:53.321878 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0fdfae-e6ec-4215-9c97-fc3234410fcb None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:53.325883 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd08a5f-8329-4c42-93fc-e90f2cf5853f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:53.326685 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd08a5f-8329-4c42-93fc-e90f2cf5853f None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:53.326971 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd08a5f-8329-4c42-93fc-e90f2cf5853f None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:53.327248 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd08a5f-8329-4c42-93fc-e90f2cf5853f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:53.335050 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd08a5f-8329-4c42-93fc-e90f2cf5853f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:53.336655 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd08a5f-8329-4c42-93fc-e90f2cf5853f None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:53.338912 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:53.351430 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-4fd08a5f-8329-4c42-93fc-e90f2cf5853f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:53.351430 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:53.351430 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:53.351430 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:53.351430 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:53.351430 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:53.351430 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:53.351430 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:53.351430 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:53.351430 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:53.351430 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:53.351430 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:53.351430 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 16:55:53.351430 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 16:55:53.351430 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 16:55:53.352680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 16:55:53.352680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:53.352680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:53.352680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:53.352680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:53.352680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:53.352680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:53.352680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:53.352680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:53.352680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:53.352680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:53.352680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:53.352680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:53.352680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:53.352680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:53.352680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:53.352680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:53.353592 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 16:55:53.353592 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 16:55:53.353592 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 16:55:53.353592 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 16:55:53.353592 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 16:55:53.353592 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 16:55:53.353592 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:53.353592 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:53.353592 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 526/1076] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:55:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:53.358186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd08a5f-8329-4c42-93fc-e90f2cf5853f None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:53.429692 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0fdfae-e6ec-4215-9c97-fc3234410fcb None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:53.462640 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd08a5f-8329-4c42-93fc-e90f2cf5853f None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:53.464877 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818f2e27-c460-4033-8e0f-0250d94875a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:53.465764 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f2e27-c460-4033-8e0f-0250d94875a6 None admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:53.466043 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f2e27-c460-4033-8e0f-0250d94875a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:53.466326 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818f2e27-c460-4033-8e0f-0250d94875a6 None admin] PATH_INFO: `/v3/projects/a0c8d8821b4b41888b7ea09ee950be20/users/a8cf1915bcb74d419cdb783bbcde8e0e/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:53.468325 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818f2e27-c460-4033-8e0f-0250d94875a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=a0c8d8821b4b41888b7ea09ee950be20, user_id=a8cf1915bcb74d419cdb783bbcde8e0e, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:53.469755 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818f2e27-c460-4033-8e0f-0250d94875a6 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:53.498810 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b6446-5ef1-4c4f-ac5b-bafa3c26216f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:53.499484 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b6446-5ef1-4c4f-ac5b-bafa3c26216f None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:53.499802 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b6446-5ef1-4c4f-ac5b-bafa3c26216f None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:53.500086 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b6446-5ef1-4c4f-ac5b-bafa3c26216f None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:53.500926 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b6446-5ef1-4c4f-ac5b-bafa3c26216f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:53.501240 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b6446-5ef1-4c4f-ac5b-bafa3c26216f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:53.502672 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b6446-5ef1-4c4f-ac5b-bafa3c26216f None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:53.515635 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 527/1077] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 6 16:55:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:53.521931 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b6446-5ef1-4c4f-ac5b-bafa3c26216f None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:53.525641 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:53.560003 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 551/1078] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 6 16:55:53 2022] PUT /identity/v3/projects/a0c8d8821b4b41888b7ea09ee950be20/users/a8cf1915bcb74d419cdb783bbcde8e0e/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 16:55:53.631957 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b6446-5ef1-4c4f-ac5b-bafa3c26216f None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:53.637266 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d37f6-64ce-44c6-ac3a-230a28108c34 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:53.637600 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d37f6-64ce-44c6-ac3a-230a28108c34 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:53.637840 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71d37f6-64ce-44c6-ac3a-230a28108c34 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:53.639358 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 552/1079] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 16:55:53.644820 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6899e6-ef96-4e4a-9d8c-d6626b5968d2 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:53.645113 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6899e6-ef96-4e4a-9d8c-d6626b5968d2 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:53.645397 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6899e6-ef96-4e4a-9d8c-d6626b5968d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:53.663334 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-6f6899e6-ef96-4e4a-9d8c-d6626b5968d2 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:53.664616 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-6f6899e6-ef96-4e4a-9d8c-d6626b5968d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:53.672328 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d145c20-cfa6-49f0-bb4b-96d6b4cb2a9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:53.673250 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d145c20-cfa6-49f0-bb4b-96d6b4cb2a9a None admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:53.673534 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d145c20-cfa6-49f0-bb4b-96d6b4cb2a9a None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:53.673809 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d145c20-cfa6-49f0-bb4b-96d6b4cb2a9a None admin] PATH_INFO: `/v3/projects/7938b0346d4240ab89774327d09a89ae/users/af26ef96dc9d47dc91549b736939268e/roles/4d3a77c1547c43aba11b60377d04d537` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:53.676346 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d145c20-cfa6-49f0-bb4b-96d6b4cb2a9a None admin] RBAC: Authorizing `identity:create_grant(project_id=7938b0346d4240ab89774327d09a89ae, user_id=af26ef96dc9d47dc91549b736939268e, role_id=4d3a77c1547c43aba11b60377d04d537)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:53.678372 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d145c20-cfa6-49f0-bb4b-96d6b4cb2a9a None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:53.736469 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 553/1080] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 16:55:53.754090 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 528/1081] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 6 16:55:53 2022] PUT /identity/v3/projects/7938b0346d4240ab89774327d09a89ae/users/af26ef96dc9d47dc91549b736939268e/roles/4d3a77c1547c43aba11b60377d04d537 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 16:55:53.760065 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6899e6-ef96-4e4a-9d8c-d6626b5968d2 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:53.764055 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:53.868682 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6899e6-ef96-4e4a-9d8c-d6626b5968d2 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:53.910409 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34e8f8-34c7-4e29-a766-e302ce33bfa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:53.911086 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34e8f8-34c7-4e29-a766-e302ce33bfa6 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:53.911369 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34e8f8-34c7-4e29-a766-e302ce33bfa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:53.911703 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea34e8f8-34c7-4e29-a766-e302ce33bfa6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:53.988088 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea34e8f8-34c7-4e29-a766-e302ce33bfa6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:53.990521 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea34e8f8-34c7-4e29-a766-e302ce33bfa6 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:53.994219 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:54.102130 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 554/1082] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:55:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 16:55:54.107850 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d145c20-cfa6-49f0-bb4b-96d6b4cb2a9a None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:54.171777 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:54.210703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d145c20-cfa6-49f0-bb4b-96d6b4cb2a9a None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:54.246636 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93804a6d-5e0d-4f70-ae24-4d1facb2a17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:54.247443 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93804a6d-5e0d-4f70-ae24-4d1facb2a17c None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:54.247883 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93804a6d-5e0d-4f70-ae24-4d1facb2a17c None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:54.248167 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93804a6d-5e0d-4f70-ae24-4d1facb2a17c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:54.259984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93804a6d-5e0d-4f70-ae24-4d1facb2a17c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:54.261484 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93804a6d-5e0d-4f70-ae24-4d1facb2a17c None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:54.280299 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-93804a6d-5e0d-4f70-ae24-4d1facb2a17c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:54.280299 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:54.280299 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:54.280299 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:54.280299 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:54.280299 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:54.280299 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:54.280299 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:54.280299 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:54.280299 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:54.280299 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:54.280299 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:54.280299 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:55:54.280299 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:55:54.280299 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:55:54.281269 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:55:54.281269 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:54.281269 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:54.281269 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:54.281269 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:54.281269 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:54.281269 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:54.281269 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:54.281269 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:54.281269 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:54.281269 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:54.281269 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:54.281269 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:54.281269 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:54.281269 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:54.281269 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:54.281269 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:54.282155 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:55:54.282155 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:55:54.282155 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:54.282155 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:54.286402 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 529/1083] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:54.287783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34e8f8-34c7-4e29-a766-e302ce33bfa6 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:54.387754 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea34e8f8-34c7-4e29-a766-e302ce33bfa6 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:54.390077 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:54.423809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55efc2fd-58dd-4ad8-94d7-8d921cec8587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:54.424400 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55efc2fd-58dd-4ad8-94d7-8d921cec8587 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:54.424779 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55efc2fd-58dd-4ad8-94d7-8d921cec8587 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:54.425064 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55efc2fd-58dd-4ad8-94d7-8d921cec8587 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:54.425843 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55efc2fd-58dd-4ad8-94d7-8d921cec8587 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:54.426309 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55efc2fd-58dd-4ad8-94d7-8d921cec8587 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:54.427784 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55efc2fd-58dd-4ad8-94d7-8d921cec8587 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:54.439443 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 555/1084] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:55:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:55.586446 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02c643-084b-4400-a70b-ff093251e1cb None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:55.586978 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02c643-084b-4400-a70b-ff093251e1cb None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:55.587102 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02c643-084b-4400-a70b-ff093251e1cb None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:55.588438 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 530/1085] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:55.593776 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79444d12-3df7-464c-b9fd-1345fb6dfaf4 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:55.594181 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79444d12-3df7-464c-b9fd-1345fb6dfaf4 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:55.594328 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79444d12-3df7-464c-b9fd-1345fb6dfaf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:55.612003 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-79444d12-3df7-464c-b9fd-1345fb6dfaf4 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:55.613174 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-79444d12-3df7-464c-b9fd-1345fb6dfaf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:55.679007 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 556/1086] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:55:55.701909 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f02c643-084b-4400-a70b-ff093251e1cb None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:55.766151 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc21d61-b1de-4c38-8f0d-25e26a10704c None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:55.766353 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc21d61-b1de-4c38-8f0d-25e26a10704c None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:55.766618 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc21d61-b1de-4c38-8f0d-25e26a10704c None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:55.768005 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 557/1087] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 16:55:55.772850 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca4855-35f7-4c01-88d3-2872e3a017c2 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:55.773127 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca4855-35f7-4c01-88d3-2872e3a017c2 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:55.773404 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca4855-35f7-4c01-88d3-2872e3a017c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:55.790195 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-a5ca4855-35f7-4c01-88d3-2872e3a017c2 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:55.791365 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-a5ca4855-35f7-4c01-88d3-2872e3a017c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:55.805944 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f02c643-084b-4400-a70b-ff093251e1cb None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:55.821786 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:55.844822 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c7923-f326-4fff-b1a9-4c867c85cee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:55.844822 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c7923-f326-4fff-b1a9-4c867c85cee6 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:55.844822 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c7923-f326-4fff-b1a9-4c867c85cee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:55.844822 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2c7923-f326-4fff-b1a9-4c867c85cee6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:55.850596 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2c7923-f326-4fff-b1a9-4c867c85cee6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:55:55.857244 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 558/1088] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 16:55:55.858264 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-3e2c7923-f326-4fff-b1a9-4c867c85cee6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:55:55.858264 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:55.858264 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:55.858264 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:55.858264 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:55.858264 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:55.858264 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:55.858264 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:55.858264 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:55.858264 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:55.858264 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:55.858264 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:55.858264 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:55:55.858264 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:55:55.858264 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:55:55.858264 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:55:55.859352 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:55:55.859352 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:55:55.859352 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:55:55.859352 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:55:55.859352 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:55.859352 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:55.859352 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:55.859352 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:55.859352 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:55.859352 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:55.859352 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:55.859352 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:55.859352 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:55.859352 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:55.859352 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:55.859352 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:55.860253 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:55.860253 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:55.860253 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:55.860253 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:55.860253 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:55:55.860253 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:55:55.860253 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:55:55.860253 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:55:55.860253 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:55:55.860253 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:55:55.860253 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:55:55.860253 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:55.860253 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 531/1089] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 6 16:55:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:55.865854 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ca4855-35f7-4c01-88d3-2872e3a017c2 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:55.882045 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c7923-f326-4fff-b1a9-4c867c85cee6 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:55.889864 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:55.963123 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ca4855-35f7-4c01-88d3-2872e3a017c2 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:55.979261 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2c7923-f326-4fff-b1a9-4c867c85cee6 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:55.996953 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829a9729-2a77-4e9d-a584-c3467f55f9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:55.997623 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829a9729-2a77-4e9d-a584-c3467f55f9a0 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:55.997910 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829a9729-2a77-4e9d-a584-c3467f55f9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:55.998191 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829a9729-2a77-4e9d-a584-c3467f55f9a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:55.998933 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829a9729-2a77-4e9d-a584-c3467f55f9a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:55.999241 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829a9729-2a77-4e9d-a584-c3467f55f9a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:56.000639 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829a9729-2a77-4e9d-a584-c3467f55f9a0 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:56.008554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 559/1090] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:55:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:56.011788 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a1cd7-8807-4e10-81a2-48b0760d29fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:56.012602 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a1cd7-8807-4e10-81a2-48b0760d29fe None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:56.012890 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a1cd7-8807-4e10-81a2-48b0760d29fe None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:56.013176 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a1cd7-8807-4e10-81a2-48b0760d29fe None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:56.014176 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829a9729-2a77-4e9d-a584-c3467f55f9a0 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:56.024406 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051a1cd7-8807-4e10-81a2-48b0760d29fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:55:56.035091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-051a1cd7-8807-4e10-81a2-48b0760d29fe None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 06 16:55:56.035091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:56.035091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:56.035091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:56.035091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:56.035091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:56.035091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:56.035091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:56.035091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:56.035091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:56.035091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:56.035091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:56.035091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:55:56.035091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:55:56.035091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:55:56.035091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:55:56.036185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:55:56.036185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:55:56.036185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:55:56.036185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:55:56.036185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:56.036185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:56.036185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:56.036185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:56.036185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:56.036185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:56.036185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:56.036185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:56.036185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:56.036185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:56.036185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:56.036185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:56.037093 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:56.037093 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:56.037093 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:56.037093 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:56.037093 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:55:56.037093 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:55:56.037093 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:55:56.037093 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:55:56.037093 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:55:56.037093 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:55:56.037093 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 06 16:55:56.037093 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:56.037093 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 532/1091] 10.222.0.40 () {56 vars in 1120 bytes} [Wed Jul 6 16:55:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:55:56.042963 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a1cd7-8807-4e10-81a2-48b0760d29fe None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:56.077462 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:56.100749 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:56.115413 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829a9729-2a77-4e9d-a584-c3467f55f9a0 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:56.142750 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a1cd7-8807-4e10-81a2-48b0760d29fe None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:56.149320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521e9bac-a680-44fd-a03d-653a00cb9eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:56.150203 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e9bac-a680-44fd-a03d-653a00cb9eee None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:56.150489 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e9bac-a680-44fd-a03d-653a00cb9eee None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:56.150768 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e9bac-a680-44fd-a03d-653a00cb9eee None admin] PATH_INFO: `/v3/users/a8cf1915bcb74d419cdb783bbcde8e0e` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:56.151994 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521e9bac-a680-44fd-a03d-653a00cb9eee None admin] RBAC: Authorizing `identity:get_user(user_id=a8cf1915bcb74d419cdb783bbcde8e0e)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:56.153432 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521e9bac-a680-44fd-a03d-653a00cb9eee None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:56.155075 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 560/1092] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:55:56 2022] GET /identity/v3/users/a8cf1915bcb74d419cdb783bbcde8e0e => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:56.160619 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521e9bac-a680-44fd-a03d-653a00cb9eee None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:56.178436 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c58cb2b-ed25-4869-9196-cea1e2fc918b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:56.179108 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c58cb2b-ed25-4869-9196-cea1e2fc918b None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:56.179391 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c58cb2b-ed25-4869-9196-cea1e2fc918b None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:56.179676 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c58cb2b-ed25-4869-9196-cea1e2fc918b None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:56.180460 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c58cb2b-ed25-4869-9196-cea1e2fc918b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:56.180773 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c58cb2b-ed25-4869-9196-cea1e2fc918b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:56.182157 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c58cb2b-ed25-4869-9196-cea1e2fc918b None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:56.190193 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 533/1093] 10.222.0.40 () {56 vars in 1115 bytes} [Wed Jul 6 16:55:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:56.195757 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c58cb2b-ed25-4869-9196-cea1e2fc918b None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:56.275222 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521e9bac-a680-44fd-a03d-653a00cb9eee None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:56.355402 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:56.359440 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c58cb2b-ed25-4869-9196-cea1e2fc918b None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:56.360241 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63477cef-dbff-4237-a50a-d0bd2a28b2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:56.360901 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63477cef-dbff-4237-a50a-d0bd2a28b2b4 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:56.361187 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63477cef-dbff-4237-a50a-d0bd2a28b2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:56.361464 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63477cef-dbff-4237-a50a-d0bd2a28b2b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:56.385998 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:56.392847 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311bd986-67cd-4bcc-8c27-625fb56c8e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:56.393799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311bd986-67cd-4bcc-8c27-625fb56c8e46 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:56.394405 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311bd986-67cd-4bcc-8c27-625fb56c8e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:56.394686 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311bd986-67cd-4bcc-8c27-625fb56c8e46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:56.405599 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311bd986-67cd-4bcc-8c27-625fb56c8e46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:56.407056 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311bd986-67cd-4bcc-8c27-625fb56c8e46 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:56.424571 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-311bd986-67cd-4bcc-8c27-625fb56c8e46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:56.424571 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:56.424571 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:56.424571 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:56.424571 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:56.424571 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:56.424571 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:56.424571 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:56.424571 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:56.424571 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:56.424571 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:56.424571 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:56.424571 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:55:56.424571 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:55:56.424571 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:55:56.425632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:55:56.425632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:56.425632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:56.425632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:56.425632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:56.425632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:56.425632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:56.425632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:56.425632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:56.425632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:56.425632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:56.425632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:56.425632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:56.425632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:56.425632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:56.425632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:56.425632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:56.426568 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:55:56.426568 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:55:56.426568 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:56.426568 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:56.426568 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 534/1094] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:56.430824 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63477cef-dbff-4237-a50a-d0bd2a28b2b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:56.431305 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311bd986-67cd-4bcc-8c27-625fb56c8e46 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:56.432315 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63477cef-dbff-4237-a50a-d0bd2a28b2b4 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:56.546290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311bd986-67cd-4bcc-8c27-625fb56c8e46 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:56.552858 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 561/1095] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:55:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 06 16:55:56.558814 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63477cef-dbff-4237-a50a-d0bd2a28b2b4 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:56.586135 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa98852-2fb4-43fe-a698-6f0aba5187fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:56.586999 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa98852-2fb4-43fe-a698-6f0aba5187fd None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:56.587565 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa98852-2fb4-43fe-a698-6f0aba5187fd None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:56.587565 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa98852-2fb4-43fe-a698-6f0aba5187fd None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:56.588303 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa98852-2fb4-43fe-a698-6f0aba5187fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:56.588617 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa98852-2fb4-43fe-a698-6f0aba5187fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:56.590380 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa98852-2fb4-43fe-a698-6f0aba5187fd None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:56.602989 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 535/1096] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:55:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:56.609777 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa98852-2fb4-43fe-a698-6f0aba5187fd None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:56.620311 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:56.621145 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:56.667885 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63477cef-dbff-4237-a50a-d0bd2a28b2b4 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:56.705196 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3aad6b-027c-4e92-9a9c-8302e284d9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:56.705891 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3aad6b-027c-4e92-9a9c-8302e284d9a7 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:56.706182 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3aad6b-027c-4e92-9a9c-8302e284d9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:56.706459 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3aad6b-027c-4e92-9a9c-8302e284d9a7 None admin] PATH_INFO: `/v3/projects/a0c8d8821b4b41888b7ea09ee950be20` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:56.707667 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3aad6b-027c-4e92-9a9c-8302e284d9a7 None admin] RBAC: Authorizing `identity:get_project(project_id=a0c8d8821b4b41888b7ea09ee950be20)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:56.709701 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3aad6b-027c-4e92-9a9c-8302e284d9a7 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:56.711559 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 562/1097] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:55:56 2022] GET /identity/v3/projects/a0c8d8821b4b41888b7ea09ee950be20 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:56.716207 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa98852-2fb4-43fe-a698-6f0aba5187fd None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:56.717582 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3aad6b-027c-4e92-9a9c-8302e284d9a7 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:56.756364 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e0152-4d17-4b81-a0e9-3838c25b2fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:56.757261 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e0152-4d17-4b81-a0e9-3838c25b2fe3 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:56.757617 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e0152-4d17-4b81-a0e9-3838c25b2fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:56.757961 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389e0152-4d17-4b81-a0e9-3838c25b2fe3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:56.766563 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389e0152-4d17-4b81-a0e9-3838c25b2fe3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:56.768046 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389e0152-4d17-4b81-a0e9-3838c25b2fe3 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:56.782254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-389e0152-4d17-4b81-a0e9-3838c25b2fe3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 06 16:55:56.782254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:56.782254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:56.782254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:56.782254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:56.782254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:56.782254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:56.782254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:56.782254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:56.782254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:56.782254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:56.782254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:56.782254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 16:55:56.782254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 16:55:56.782254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 16:55:56.782254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 16:55:56.783250 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:56.783250 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:56.783250 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 16:55:56.783250 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:56.783250 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 16:55:56.783250 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:55:56.783250 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:56.783250 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:56.783250 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:56.783250 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:56.783250 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:56.783250 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:56.783250 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:56.783250 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:56.783250 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:56.783250 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:56.783250 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:56.783250 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:56.784433 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:56.784433 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:56.784433 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 16:55:56.784433 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 16:55:56.784433 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 16:55:56.784433 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 16:55:56.784433 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 16:55:56.784433 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 16:55:56.784433 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 06 16:55:56.784433 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:56.784433 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 536/1098] 10.222.0.40 () {56 vars in 1093 bytes} [Wed Jul 6 16:55:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:55:56.789239 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e0152-4d17-4b81-a0e9-3838c25b2fe3 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:56.819594 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3aad6b-027c-4e92-9a9c-8302e284d9a7 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:56.841029 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:56.852224 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c14a59-7310-481b-81c5-b7084b982c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:56.852876 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c14a59-7310-481b-81c5-b7084b982c5b None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:56.853159 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c14a59-7310-481b-81c5-b7084b982c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:56.853434 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c14a59-7310-481b-81c5-b7084b982c5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:56.854256 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c14a59-7310-481b-81c5-b7084b982c5b None admin] RBAC: Adding query filter params (role.id=b41f8a7a58e64f749f2099ca45cc04a7, scope.project.id=a0c8d8821b4b41888b7ea09ee950be20, user.id=a8cf1915bcb74d419cdb783bbcde8e0e) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:56.854565 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c14a59-7310-481b-81c5-b7084b982c5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:56.856047 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c14a59-7310-481b-81c5-b7084b982c5b None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:56.856582 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:56.861734 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 563/1099] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:55:56 2022] GET /identity/v3/role_assignments?user.id=a8cf1915bcb74d419cdb783bbcde8e0e&scope.project.id=a0c8d8821b4b41888b7ea09ee950be20&role.id=b41f8a7a58e64f749f2099ca45cc04a7 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:56.891844 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389e0152-4d17-4b81-a0e9-3838c25b2fe3 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:56.928382 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d2a87-f6b4-46fe-bc60-0159adffe1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:56.929062 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d2a87-f6b4-46fe-bc60-0159adffe1d7 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:56.929346 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d2a87-f6b4-46fe-bc60-0159adffe1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:56.929621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80d2a87-f6b4-46fe-bc60-0159adffe1d7 None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:56.930470 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80d2a87-f6b4-46fe-bc60-0159adffe1d7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:56.930782 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80d2a87-f6b4-46fe-bc60-0159adffe1d7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:56.932290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80d2a87-f6b4-46fe-bc60-0159adffe1d7 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:56.952685 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 537/1100] 10.222.0.40 () {56 vars in 1133 bytes} [Wed Jul 6 16:55:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:55:56.958661 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c14a59-7310-481b-81c5-b7084b982c5b None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:57.060408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c14a59-7310-481b-81c5-b7084b982c5b None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:57.093241 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4a8026-bfa8-4b53-b855-fdd233e7b4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:57.093891 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a8026-bfa8-4b53-b855-fdd233e7b4fa None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:57.094176 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a8026-bfa8-4b53-b855-fdd233e7b4fa None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:57.094452 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4a8026-bfa8-4b53-b855-fdd233e7b4fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:57.147466 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:57.160066 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4a8026-bfa8-4b53-b855-fdd233e7b4fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:57.161603 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4a8026-bfa8-4b53-b855-fdd233e7b4fa None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:57.260046 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 564/1101] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:55:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 06 16:55:57.265730 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d2a87-f6b4-46fe-bc60-0159adffe1d7 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:57.404857 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80d2a87-f6b4-46fe-bc60-0159adffe1d7 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:57.430604 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:57.438123 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f456b-0301-4780-8f6e-d067f42d6b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:57.438909 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f456b-0301-4780-8f6e-d067f42d6b1d None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:57.444236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f456b-0301-4780-8f6e-d067f42d6b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:57.444236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f456b-0301-4780-8f6e-d067f42d6b1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:57.449795 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f456b-0301-4780-8f6e-d067f42d6b1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:57.451229 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f456b-0301-4780-8f6e-d067f42d6b1d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:57.469203 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-789f456b-0301-4780-8f6e-d067f42d6b1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:57.469203 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:57.469203 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:57.469203 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:57.469203 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:57.469203 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:57.469203 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:57.469203 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:57.469203 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:57.469203 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:57.469203 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:57.469203 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:57.469203 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:55:57.469203 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:55:57.469203 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:55:57.470254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:55:57.470254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:57.470254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:57.470254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:57.470254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:57.470254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:57.470254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:57.470254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:57.470254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:57.470254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:57.470254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:57.470254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:57.470254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:57.470254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:57.470254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:57.470254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:57.470254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:57.471172 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:55:57.471172 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:55:57.471172 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:55:57.471172 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:57.471172 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 538/1102] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:57.476084 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4a8026-bfa8-4b53-b855-fdd233e7b4fa None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:57.558683 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:57.574861 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4a8026-bfa8-4b53-b855-fdd233e7b4fa None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:57.610930 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da49373-b46b-4391-a5bb-a6f1a246510f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:57.611593 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da49373-b46b-4391-a5bb-a6f1a246510f None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:57.611908 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da49373-b46b-4391-a5bb-a6f1a246510f None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:57.612184 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da49373-b46b-4391-a5bb-a6f1a246510f None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:57.612897 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da49373-b46b-4391-a5bb-a6f1a246510f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:57.613200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da49373-b46b-4391-a5bb-a6f1a246510f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:57.614549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da49373-b46b-4391-a5bb-a6f1a246510f None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:57.625614 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 565/1103] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:55:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:57.631244 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f456b-0301-4780-8f6e-d067f42d6b1d None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:57.728148 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f456b-0301-4780-8f6e-d067f42d6b1d None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:57.761302 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04de4be-28b6-40f1-8800-291617b7a808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:57.762085 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04de4be-28b6-40f1-8800-291617b7a808 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:57.762366 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04de4be-28b6-40f1-8800-291617b7a808 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:57.762638 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04de4be-28b6-40f1-8800-291617b7a808 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:57.769113 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04de4be-28b6-40f1-8800-291617b7a808 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:57.770568 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04de4be-28b6-40f1-8800-291617b7a808 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:57.784542 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-b04de4be-28b6-40f1-8800-291617b7a808 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:57.784542 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:55:57.784542 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:55:57.784542 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:55:57.784542 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:55:57.784542 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:55:57.784542 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:55:57.784542 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:55:57.784542 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:55:57.784542 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:55:57.784542 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:55:57.784542 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:55:57.784542 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 16:55:57.784542 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 16:55:57.784542 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 16:55:57.785569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 16:55:57.785569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:55:57.785569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:55:57.785569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:55:57.785569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:55:57.785569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:55:57.785569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:55:57.785569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:55:57.785569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:55:57.785569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:55:57.785569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:55:57.785569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:55:57.785569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:55:57.785569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:55:57.785569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:55:57.785569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:55:57.785569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:55:57.786477 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 16:55:57.786477 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 16:55:57.786477 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 16:55:57.786477 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 16:55:57.786477 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 16:55:57.786477 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 16:55:57.786477 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:55:57.786477 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:55:57.786477 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 539/1104] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:55:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:55:57.792032 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da49373-b46b-4391-a5bb-a6f1a246510f None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:57.889175 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da49373-b46b-4391-a5bb-a6f1a246510f None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:57.923684 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea1e56a-dcc1-4539-8188-4369123b85af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:57.924862 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1e56a-dcc1-4539-8188-4369123b85af None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:57.925358 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1e56a-dcc1-4539-8188-4369123b85af None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:57.925835 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea1e56a-dcc1-4539-8188-4369123b85af None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:57.927363 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea1e56a-dcc1-4539-8188-4369123b85af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:57.927931 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea1e56a-dcc1-4539-8188-4369123b85af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:57.930266 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea1e56a-dcc1-4539-8188-4369123b85af None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:57.943775 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 566/1105] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 6 16:55:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:57.949503 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04de4be-28b6-40f1-8800-291617b7a808 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:57.990434 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:58.055163 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04de4be-28b6-40f1-8800-291617b7a808 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:58.088656 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db4636-24a5-4fe0-87bf-583dd392c5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:58.089306 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db4636-24a5-4fe0-87bf-583dd392c5e7 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:58.089590 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db4636-24a5-4fe0-87bf-583dd392c5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:58.089866 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db4636-24a5-4fe0-87bf-583dd392c5e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:58.090678 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4db4636-24a5-4fe0-87bf-583dd392c5e7 None admin] RBAC: Adding query filter params (role.id=4d3a77c1547c43aba11b60377d04d537, scope.project.id=7938b0346d4240ab89774327d09a89ae, user.id=af26ef96dc9d47dc91549b736939268e) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:55:58.090988 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4db4636-24a5-4fe0-87bf-583dd392c5e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:58.092438 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4db4636-24a5-4fe0-87bf-583dd392c5e7 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:58.098062 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 540/1106] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:55:57 2022] GET /identity/v3/role_assignments?user.id=af26ef96dc9d47dc91549b736939268e&scope.project.id=7938b0346d4240ab89774327d09a89ae&role.id=4d3a77c1547c43aba11b60377d04d537 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:58.922922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f62119-2e05-4687-bc7b-4850de253691 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:58.923450 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f62119-2e05-4687-bc7b-4850de253691 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:58.923450 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f62119-2e05-4687-bc7b-4850de253691 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:58.924854 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 567/1107] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:55:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:55:58.930420 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b34eb8d-600b-4ba7-8ca3-ac6678d2d228 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:58.930860 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b34eb8d-600b-4ba7-8ca3-ac6678d2d228 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:58.930983 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b34eb8d-600b-4ba7-8ca3-ac6678d2d228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:58.948060 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-1b34eb8d-600b-4ba7-8ca3-ac6678d2d228 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:55:58.949455 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-1b34eb8d-600b-4ba7-8ca3-ac6678d2d228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:55:59.019397 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 541/1108] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:55:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:55:59.044593 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f62119-2e05-4687-bc7b-4850de253691 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:59.053330 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:59.140816 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f62119-2e05-4687-bc7b-4850de253691 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:59.173425 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13705e1b-42e0-4c53-a4d8-85c9fc3f03f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:59.174190 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13705e1b-42e0-4c53-a4d8-85c9fc3f03f5 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:59.174470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13705e1b-42e0-4c53-a4d8-85c9fc3f03f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:59.174742 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13705e1b-42e0-4c53-a4d8-85c9fc3f03f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:59.175914 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13705e1b-42e0-4c53-a4d8-85c9fc3f03f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:59.177355 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13705e1b-42e0-4c53-a4d8-85c9fc3f03f5 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:59.178927 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 568/1109] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:55:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:55:59.184532 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b34eb8d-600b-4ba7-8ca3-ac6678d2d228 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:55:59.217005 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:55:59.285919 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b34eb8d-600b-4ba7-8ca3-ac6678d2d228 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:55:59.318821 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc2b5e1-09ae-4f24-bf30-66fa415d7986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:55:59.319489 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc2b5e1-09ae-4f24-bf30-66fa415d7986 None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:55:59.319802 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc2b5e1-09ae-4f24-bf30-66fa415d7986 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:55:59.320087 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc2b5e1-09ae-4f24-bf30-66fa415d7986 None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:55:59.320789 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc2b5e1-09ae-4f24-bf30-66fa415d7986 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:55:59.322224 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc2b5e1-09ae-4f24-bf30-66fa415d7986 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:55:59.408531 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 542/1110] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 6 16:55:59 2022] POST /identity/v3/users => generated 286 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 16:56:01.454453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0e0511-e062-44d7-8a16-9bfbc0482ac4 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:01.454988 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0e0511-e062-44d7-8a16-9bfbc0482ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:01.455114 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0e0511-e062-44d7-8a16-9bfbc0482ac4 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:01.456687 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 569/1111] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:56:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:56:01.461864 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70218394-479b-4daa-af4a-cad10accca30 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:01.462328 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70218394-479b-4daa-af4a-cad10accca30 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:01.462422 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70218394-479b-4daa-af4a-cad10accca30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:01.479334 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-70218394-479b-4daa-af4a-cad10accca30 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:56:01.480568 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-70218394-479b-4daa-af4a-cad10accca30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:56:01.548555 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 543/1112] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:56:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:56:01.577377 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0e0511-e062-44d7-8a16-9bfbc0482ac4 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:01.660051 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:01.687609 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0e0511-e062-44d7-8a16-9bfbc0482ac4 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:01.720888 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef68d5f1-6efe-4181-8b16-db28316562ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:01.721679 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef68d5f1-6efe-4181-8b16-db28316562ea None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:01.721961 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef68d5f1-6efe-4181-8b16-db28316562ea None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:01.722234 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef68d5f1-6efe-4181-8b16-db28316562ea None admin] PATH_INFO: `/v3/roles/c31c5ee7ed8646e48fa81e85d10fcaf2` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:01.723712 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef68d5f1-6efe-4181-8b16-db28316562ea None admin] RBAC: Authorizing `identity:get_role(role_id=c31c5ee7ed8646e48fa81e85d10fcaf2)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:01.725647 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef68d5f1-6efe-4181-8b16-db28316562ea None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:01.727024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 570/1113] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:56:01 2022] GET /identity/v3/roles/c31c5ee7ed8646e48fa81e85d10fcaf2 => generated 223 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:56:01.733193 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70218394-479b-4daa-af4a-cad10accca30 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:01.754636 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:01.832032 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70218394-479b-4daa-af4a-cad10accca30 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:01.864485 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b74ce0e-121a-4fbb-a391-b1e93f9c7684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:01.865909 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74ce0e-121a-4fbb-a391-b1e93f9c7684 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:01.866187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74ce0e-121a-4fbb-a391-b1e93f9c7684 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:01.866458 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74ce0e-121a-4fbb-a391-b1e93f9c7684 None admin] PATH_INFO: `/v3/users/09de396fe883476d9290e81404af0fea` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:01.879088 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b74ce0e-121a-4fbb-a391-b1e93f9c7684 None admin] RBAC: Authorizing `identity:get_user(user_id=09de396fe883476d9290e81404af0fea)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:01.880636 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b74ce0e-121a-4fbb-a391-b1e93f9c7684 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:01.882301 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 544/1114] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:56:01 2022] GET /identity/v3/users/09de396fe883476d9290e81404af0fea => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:56:01.888078 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef68d5f1-6efe-4181-8b16-db28316562ea None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:01.989595 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef68d5f1-6efe-4181-8b16-db28316562ea None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:02.022607 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a410431-3d54-4916-9905-39f0461906d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:02.023251 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a410431-3d54-4916-9905-39f0461906d2 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:02.023536 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a410431-3d54-4916-9905-39f0461906d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:02.023847 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a410431-3d54-4916-9905-39f0461906d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:02.035208 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:02.090182 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a410431-3d54-4916-9905-39f0461906d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:02.091635 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a410431-3d54-4916-9905-39f0461906d2 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:02.186509 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 571/1115] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:56:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 16:56:02.192391 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b74ce0e-121a-4fbb-a391-b1e93f9c7684 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:02.267532 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:02.290992 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b74ce0e-121a-4fbb-a391-b1e93f9c7684 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:02.329088 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a043dd8-3205-4fa8-8e09-39ea0eefe1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:02.329941 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a043dd8-3205-4fa8-8e09-39ea0eefe1e6 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:02.330224 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a043dd8-3205-4fa8-8e09-39ea0eefe1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:02.330497 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a043dd8-3205-4fa8-8e09-39ea0eefe1e6 None admin] PATH_INFO: `/v3/projects/a0c8d8821b4b41888b7ea09ee950be20` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:02.331695 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a043dd8-3205-4fa8-8e09-39ea0eefe1e6 None admin] RBAC: Authorizing `identity:get_project(project_id=a0c8d8821b4b41888b7ea09ee950be20)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:02.333138 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a043dd8-3205-4fa8-8e09-39ea0eefe1e6 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:02.335390 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 545/1116] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:56:02 2022] GET /identity/v3/projects/a0c8d8821b4b41888b7ea09ee950be20 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:56:02.341571 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a410431-3d54-4916-9905-39f0461906d2 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:02.385000 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:02.438467 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a410431-3d54-4916-9905-39f0461906d2 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:02.472030 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0464243-5ad7-4db1-8446-3e83aa0ed2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:02.472675 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0464243-5ad7-4db1-8446-3e83aa0ed2c7 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:02.472959 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0464243-5ad7-4db1-8446-3e83aa0ed2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:02.473234 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0464243-5ad7-4db1-8446-3e83aa0ed2c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:02.474076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0464243-5ad7-4db1-8446-3e83aa0ed2c7 None admin] RBAC: Adding query filter params (role.id=c31c5ee7ed8646e48fa81e85d10fcaf2, scope.project.id=a0c8d8821b4b41888b7ea09ee950be20, user.id=09de396fe883476d9290e81404af0fea) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:56:02.474387 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0464243-5ad7-4db1-8446-3e83aa0ed2c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:02.475815 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0464243-5ad7-4db1-8446-3e83aa0ed2c7 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:02.481091 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 572/1117] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:56:02 2022] GET /identity/v3/role_assignments?user.id=09de396fe883476d9290e81404af0fea&scope.project.id=a0c8d8821b4b41888b7ea09ee950be20&role.id=c31c5ee7ed8646e48fa81e85d10fcaf2 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:56:04.490200 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c197c2-3caa-4328-8dd5-5f077448127b None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:04.490789 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c197c2-3caa-4328-8dd5-5f077448127b None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:04.490789 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c197c2-3caa-4328-8dd5-5f077448127b None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:04.492131 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 546/1118] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:56:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:56:04.497245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e55768-a369-4fa4-913d-22751c93464c None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:04.497687 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e55768-a369-4fa4-913d-22751c93464c None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:04.497790 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e55768-a369-4fa4-913d-22751c93464c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:04.514753 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-69e55768-a369-4fa4-913d-22751c93464c None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:56:04.515964 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-69e55768-a369-4fa4-913d-22751c93464c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:56:04.580021 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 573/1119] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:56:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 16:56:04.603852 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c197c2-3caa-4328-8dd5-5f077448127b None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:04.699922 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c197c2-3caa-4328-8dd5-5f077448127b None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:04.732957 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67923628-b5e9-4c76-80ba-00dfdd2f3064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:04.733281 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67923628-b5e9-4c76-80ba-00dfdd2f3064 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:04.733564 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67923628-b5e9-4c76-80ba-00dfdd2f3064 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:04.733837 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67923628-b5e9-4c76-80ba-00dfdd2f3064 None admin] PATH_INFO: `/v3/roles/c31c5ee7ed8646e48fa81e85d10fcaf2` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:04.735301 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67923628-b5e9-4c76-80ba-00dfdd2f3064 None admin] RBAC: Authorizing `identity:get_role(role_id=c31c5ee7ed8646e48fa81e85d10fcaf2)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:04.736695 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67923628-b5e9-4c76-80ba-00dfdd2f3064 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:04.738057 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 547/1120] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:56:04 2022] GET /identity/v3/roles/c31c5ee7ed8646e48fa81e85d10fcaf2 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:56:04.743850 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e55768-a369-4fa4-913d-22751c93464c None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:04.778054 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:04.840012 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e55768-a369-4fa4-913d-22751c93464c None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:04.871923 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9fcef9-a38c-420b-bea6-c878813552ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:04.872784 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9fcef9-a38c-420b-bea6-c878813552ab None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:04.873066 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9fcef9-a38c-420b-bea6-c878813552ab None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:04.873336 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9fcef9-a38c-420b-bea6-c878813552ab None admin] PATH_INFO: `/v3/users/09de396fe883476d9290e81404af0fea` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:04.874502 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9fcef9-a38c-420b-bea6-c878813552ab None admin] RBAC: Authorizing `identity:get_user(user_id=09de396fe883476d9290e81404af0fea)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:04.875953 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9fcef9-a38c-420b-bea6-c878813552ab None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:04.877543 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 574/1121] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:56:04 2022] GET /identity/v3/users/09de396fe883476d9290e81404af0fea => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:56:04.883719 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67923628-b5e9-4c76-80ba-00dfdd2f3064 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:04.899876 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:04.981160 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67923628-b5e9-4c76-80ba-00dfdd2f3064 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:05.014417 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841f38f6-5d00-43ee-84c2-cf1aaa09c304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:05.015215 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f38f6-5d00-43ee-84c2-cf1aaa09c304 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:05.015503 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f38f6-5d00-43ee-84c2-cf1aaa09c304 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:05.015807 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841f38f6-5d00-43ee-84c2-cf1aaa09c304 None admin] PATH_INFO: `/v3/projects/a0c8d8821b4b41888b7ea09ee950be20` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:05.016970 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841f38f6-5d00-43ee-84c2-cf1aaa09c304 None admin] RBAC: Authorizing `identity:get_project(project_id=a0c8d8821b4b41888b7ea09ee950be20)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:05.018399 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841f38f6-5d00-43ee-84c2-cf1aaa09c304 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:05.020109 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 548/1122] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:56:04 2022] GET /identity/v3/projects/a0c8d8821b4b41888b7ea09ee950be20 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:56:05.026305 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9fcef9-a38c-420b-bea6-c878813552ab None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:05.121622 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9fcef9-a38c-420b-bea6-c878813552ab None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:05.123893 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:05.155305 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b203e3ba-6fec-435e-a06d-cd32fc391db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:05.156284 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203e3ba-6fec-435e-a06d-cd32fc391db7 None admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:05.156573 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203e3ba-6fec-435e-a06d-cd32fc391db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:05.156847 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b203e3ba-6fec-435e-a06d-cd32fc391db7 None admin] PATH_INFO: `/v3/projects/a0c8d8821b4b41888b7ea09ee950be20/users/09de396fe883476d9290e81404af0fea/roles/c31c5ee7ed8646e48fa81e85d10fcaf2` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:05.159014 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b203e3ba-6fec-435e-a06d-cd32fc391db7 None admin] RBAC: Authorizing `identity:create_grant(project_id=a0c8d8821b4b41888b7ea09ee950be20, user_id=09de396fe883476d9290e81404af0fea, role_id=c31c5ee7ed8646e48fa81e85d10fcaf2)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:05.160457 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b203e3ba-6fec-435e-a06d-cd32fc391db7 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:05.232090 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 575/1123] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 6 16:56:05 2022] PUT /identity/v3/projects/a0c8d8821b4b41888b7ea09ee950be20/users/09de396fe883476d9290e81404af0fea/roles/c31c5ee7ed8646e48fa81e85d10fcaf2 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 16:56:07.202456 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824f4eb-f085-4224-9d34-16ed4b3f47ad None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:07.202992 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824f4eb-f085-4224-9d34-16ed4b3f47ad None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:07.202992 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824f4eb-f085-4224-9d34-16ed4b3f47ad None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:07.204416 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 549/1124] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:56:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:56:07.209330 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e22692-ac39-47ed-920a-085066ca61a7 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:07.209701 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e22692-ac39-47ed-920a-085066ca61a7 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:07.209892 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e22692-ac39-47ed-920a-085066ca61a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:07.226609 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-54e22692-ac39-47ed-920a-085066ca61a7 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:56:07.227809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-54e22692-ac39-47ed-920a-085066ca61a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:56:07.292178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 576/1125] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:56:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:56:07.314595 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a824f4eb-f085-4224-9d34-16ed4b3f47ad None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:07.392838 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:07.411873 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a824f4eb-f085-4224-9d34-16ed4b3f47ad None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:07.444290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319d7b8d-d7cc-43d2-a046-75f1b5210e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:07.445126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319d7b8d-d7cc-43d2-a046-75f1b5210e4f None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:07.445442 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319d7b8d-d7cc-43d2-a046-75f1b5210e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:07.445710 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319d7b8d-d7cc-43d2-a046-75f1b5210e4f None admin] PATH_INFO: `/v3/roles/c31c5ee7ed8646e48fa81e85d10fcaf2` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:07.447146 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319d7b8d-d7cc-43d2-a046-75f1b5210e4f None admin] RBAC: Authorizing `identity:get_role(role_id=c31c5ee7ed8646e48fa81e85d10fcaf2)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:07.448525 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319d7b8d-d7cc-43d2-a046-75f1b5210e4f None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:07.449853 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 550/1126] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:56:07 2022] GET /identity/v3/roles/c31c5ee7ed8646e48fa81e85d10fcaf2 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:56:07.455484 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e22692-ac39-47ed-920a-085066ca61a7 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:07.545450 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:07.551907 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e22692-ac39-47ed-920a-085066ca61a7 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:07.586034 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729b5f19-3d5e-4022-980e-21ef3d504335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:07.586893 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b5f19-3d5e-4022-980e-21ef3d504335 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:07.587178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b5f19-3d5e-4022-980e-21ef3d504335 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:07.587456 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b5f19-3d5e-4022-980e-21ef3d504335 None admin] PATH_INFO: `/v3/users/09de396fe883476d9290e81404af0fea` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:07.588782 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729b5f19-3d5e-4022-980e-21ef3d504335 None admin] RBAC: Authorizing `identity:get_user(user_id=09de396fe883476d9290e81404af0fea)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:07.590458 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729b5f19-3d5e-4022-980e-21ef3d504335 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:07.592019 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 577/1127] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:56:07 2022] GET /identity/v3/users/09de396fe883476d9290e81404af0fea => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:56:07.597547 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319d7b8d-d7cc-43d2-a046-75f1b5210e4f None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:07.694299 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319d7b8d-d7cc-43d2-a046-75f1b5210e4f None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:07.727337 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd56247a-cee2-4586-865b-ef9560666369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:07.727970 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56247a-cee2-4586-865b-ef9560666369 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:07.728257 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56247a-cee2-4586-865b-ef9560666369 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:07.728536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd56247a-cee2-4586-865b-ef9560666369 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:07.746457 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:07.794146 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd56247a-cee2-4586-865b-ef9560666369 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:07.795583 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd56247a-cee2-4586-865b-ef9560666369 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:07.888228 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 551/1128] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:56:07 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 16:56:07.894143 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729b5f19-3d5e-4022-980e-21ef3d504335 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:07.988938 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729b5f19-3d5e-4022-980e-21ef3d504335 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:08.022574 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6296f277-5626-447f-bd5a-ae53a6414359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:08.023414 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6296f277-5626-447f-bd5a-ae53a6414359 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:08.023697 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6296f277-5626-447f-bd5a-ae53a6414359 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:08.024005 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6296f277-5626-447f-bd5a-ae53a6414359 None admin] PATH_INFO: `/v3/projects/a0c8d8821b4b41888b7ea09ee950be20` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:08.025171 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6296f277-5626-447f-bd5a-ae53a6414359 None admin] RBAC: Authorizing `identity:get_project(project_id=a0c8d8821b4b41888b7ea09ee950be20)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:08.026610 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6296f277-5626-447f-bd5a-ae53a6414359 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:08.028324 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 578/1129] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:56:07 2022] GET /identity/v3/projects/a0c8d8821b4b41888b7ea09ee950be20 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:56:08.034626 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd56247a-cee2-4586-865b-ef9560666369 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:08.081522 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:08.129323 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd56247a-cee2-4586-865b-ef9560666369 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:08.163014 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd87e2d-507f-432c-800d-ab046fcba238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:08.163662 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd87e2d-507f-432c-800d-ab046fcba238 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:08.163976 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd87e2d-507f-432c-800d-ab046fcba238 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:08.164255 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd87e2d-507f-432c-800d-ab046fcba238 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:08.165081 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd87e2d-507f-432c-800d-ab046fcba238 None admin] RBAC: Adding query filter params (role.id=c31c5ee7ed8646e48fa81e85d10fcaf2, scope.project.id=a0c8d8821b4b41888b7ea09ee950be20, user.id=09de396fe883476d9290e81404af0fea) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:56:08.165418 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd87e2d-507f-432c-800d-ab046fcba238 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:08.166842 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd87e2d-507f-432c-800d-ab046fcba238 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:08.172518 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 552/1130] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:56:08 2022] GET /identity/v3/role_assignments?user.id=09de396fe883476d9290e81404af0fea&scope.project.id=a0c8d8821b4b41888b7ea09ee950be20&role.id=c31c5ee7ed8646e48fa81e85d10fcaf2 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:56:10.168552 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f41b8a-c5d9-4869-8739-5ecb02ad761f None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:10.169092 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f41b8a-c5d9-4869-8739-5ecb02ad761f None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:10.169092 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f41b8a-c5d9-4869-8739-5ecb02ad761f None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:10.170457 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 579/1131] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:56:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:56:10.175599 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c28248-8b78-488e-920b-66d34752cdec None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:10.176049 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c28248-8b78-488e-920b-66d34752cdec None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:10.176191 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c28248-8b78-488e-920b-66d34752cdec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:10.192942 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-58c28248-8b78-488e-920b-66d34752cdec None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:56:10.194099 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-58c28248-8b78-488e-920b-66d34752cdec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:56:10.257690 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 553/1132] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:56:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:56:10.281804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f41b8a-c5d9-4869-8739-5ecb02ad761f None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:10.311769 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:10.380491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f41b8a-c5d9-4869-8739-5ecb02ad761f None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:10.414014 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71801c8-747e-44fa-b7d4-5c40d2d53eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:10.414795 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71801c8-747e-44fa-b7d4-5c40d2d53eca None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:10.415079 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71801c8-747e-44fa-b7d4-5c40d2d53eca None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:10.415352 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71801c8-747e-44fa-b7d4-5c40d2d53eca None admin] PATH_INFO: `/v3/domains/default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:10.416532 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71801c8-747e-44fa-b7d4-5c40d2d53eca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:10.417954 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71801c8-747e-44fa-b7d4-5c40d2d53eca None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:10.419611 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 580/1133] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:56:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:56:10.425351 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c28248-8b78-488e-920b-66d34752cdec None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:10.468546 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:10.520287 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c28248-8b78-488e-920b-66d34752cdec None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:10.553191 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95176b25-eaa8-42b7-b702-5896ff02239b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:10.553844 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95176b25-eaa8-42b7-b702-5896ff02239b None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:10.554131 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95176b25-eaa8-42b7-b702-5896ff02239b None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:10.554409 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95176b25-eaa8-42b7-b702-5896ff02239b None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:10.555102 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95176b25-eaa8-42b7-b702-5896ff02239b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:10.556559 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95176b25-eaa8-42b7-b702-5896ff02239b None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:10.684919 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 554/1134] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Jul 6 16:56:10 2022] POST /identity/v3/projects => generated 311 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 16:56:12.749002 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c3315-1b66-4922-9420-02d4f09e1d87 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:12.749415 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c3315-1b66-4922-9420-02d4f09e1d87 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:12.749566 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c3315-1b66-4922-9420-02d4f09e1d87 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:12.750931 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 581/1135] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:56:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:56:12.756119 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21def930-e831-4127-b7d7-5adc6ecdceee None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:12.756556 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21def930-e831-4127-b7d7-5adc6ecdceee None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:12.756660 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21def930-e831-4127-b7d7-5adc6ecdceee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:12.773454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-21def930-e831-4127-b7d7-5adc6ecdceee None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:56:12.774611 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-21def930-e831-4127-b7d7-5adc6ecdceee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:56:12.837745 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 555/1136] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:56:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 16:56:12.862274 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c3315-1b66-4922-9420-02d4f09e1d87 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:12.965458 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:12.970266 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c3315-1b66-4922-9420-02d4f09e1d87 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:13.002922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb85579-8874-4b2d-8bba-6b855269ffb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:13.003643 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb85579-8874-4b2d-8bba-6b855269ffb0 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:13.003959 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb85579-8874-4b2d-8bba-6b855269ffb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:13.004240 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb85579-8874-4b2d-8bba-6b855269ffb0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:13.005958 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb85579-8874-4b2d-8bba-6b855269ffb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:13.007397 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb85579-8874-4b2d-8bba-6b855269ffb0 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:13.009059 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 582/1137] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:56:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:56:13.014790 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21def930-e831-4127-b7d7-5adc6ecdceee None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:13.041359 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:13.110943 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21def930-e831-4127-b7d7-5adc6ecdceee None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:13.143648 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9556565c-7fc8-461e-92b7-c0c41b4620f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:13.144337 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9556565c-7fc8-461e-92b7-c0c41b4620f8 None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:13.144619 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9556565c-7fc8-461e-92b7-c0c41b4620f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:13.144895 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9556565c-7fc8-461e-92b7-c0c41b4620f8 None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:13.146460 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9556565c-7fc8-461e-92b7-c0c41b4620f8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:13.147907 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9556565c-7fc8-461e-92b7-c0c41b4620f8 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:13.265657 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 556/1138] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 6 16:56:13 2022] POST /identity/v3/users => generated 286 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 16:56:15.309238 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fb002a-a04c-4294-8d26-506bda43ca58 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:15.309768 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fb002a-a04c-4294-8d26-506bda43ca58 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:15.309860 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fb002a-a04c-4294-8d26-506bda43ca58 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:15.311156 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 583/1139] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:56:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:56:15.316210 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ead22-0031-4d8e-ac67-daf8fcb7af26 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:15.316653 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ead22-0031-4d8e-ac67-daf8fcb7af26 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:15.316757 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47ead22-0031-4d8e-ac67-daf8fcb7af26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:15.333002 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-d47ead22-0031-4d8e-ac67-daf8fcb7af26 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:56:15.334174 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d47ead22-0031-4d8e-ac67-daf8fcb7af26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:56:15.398425 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 557/1140] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:56:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 16:56:15.422425 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fb002a-a04c-4294-8d26-506bda43ca58 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:15.519499 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fb002a-a04c-4294-8d26-506bda43ca58 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:15.552745 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856347f-0b7d-48ff-b657-b819c59f0df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:15.553519 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856347f-0b7d-48ff-b657-b819c59f0df6 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:15.553805 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856347f-0b7d-48ff-b657-b819c59f0df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:15.554084 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9856347f-0b7d-48ff-b657-b819c59f0df6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:15.564005 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9856347f-0b7d-48ff-b657-b819c59f0df6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:56:15.571508 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-9856347f-0b7d-48ff-b657-b819c59f0df6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:56:15.571508 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:56:15.571508 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:56:15.571508 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:56:15.571508 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:56:15.571508 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:56:15.571508 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:56:15.571508 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:56:15.571508 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:56:15.571508 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:56:15.571508 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:56:15.571508 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:56:15.571508 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:56:15.571508 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:56:15.571508 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:56:15.571508 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:56:15.572690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:56:15.572690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:56:15.572690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:56:15.572690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:56:15.572690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:56:15.572690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:56:15.572690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:56:15.572690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:56:15.572690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:56:15.572690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:56:15.572690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:56:15.572690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:56:15.572690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:56:15.572690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:56:15.572690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:56:15.572690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:56:15.573793 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:56:15.573793 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:56:15.573793 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:56:15.573793 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:56:15.573793 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:56:15.573793 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:56:15.573793 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:56:15.573793 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:56:15.573793 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:56:15.573793 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:56:15.573793 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:56:15.573793 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:56:15.575874 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 584/1141] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 6 16:56:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:56:15.578638 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47ead22-0031-4d8e-ac67-daf8fcb7af26 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:15.592791 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:15.681815 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47ead22-0031-4d8e-ac67-daf8fcb7af26 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:15.714180 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eff049-39ac-4e7e-a644-2c78edd4246f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:15.714841 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eff049-39ac-4e7e-a644-2c78edd4246f None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:15.715126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eff049-39ac-4e7e-a644-2c78edd4246f None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:15.715403 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eff049-39ac-4e7e-a644-2c78edd4246f None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:15.716257 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eff049-39ac-4e7e-a644-2c78edd4246f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:56:15.716569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eff049-39ac-4e7e-a644-2c78edd4246f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:15.717945 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eff049-39ac-4e7e-a644-2c78edd4246f None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:15.726582 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 558/1142] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:56:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:56:15.732488 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856347f-0b7d-48ff-b657-b819c59f0df6 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:15.759249 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:15.828267 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9856347f-0b7d-48ff-b657-b819c59f0df6 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:15.861425 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7dbb0-d1b0-4296-9471-fac79aa04e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:15.862290 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7dbb0-d1b0-4296-9471-fac79aa04e36 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:15.862574 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7dbb0-d1b0-4296-9471-fac79aa04e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:15.862850 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7dbb0-d1b0-4296-9471-fac79aa04e36 None admin] PATH_INFO: `/v3/users/545bb71036bf45bfae893ac34e20a2f1` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:15.876336 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad7dbb0-d1b0-4296-9471-fac79aa04e36 None admin] RBAC: Authorizing `identity:get_user(user_id=545bb71036bf45bfae893ac34e20a2f1)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:15.877773 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad7dbb0-d1b0-4296-9471-fac79aa04e36 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:15.879397 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 585/1143] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:56:15 2022] GET /identity/v3/users/545bb71036bf45bfae893ac34e20a2f1 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:56:15.885546 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eff049-39ac-4e7e-a644-2c78edd4246f None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:15.958536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:15.983937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eff049-39ac-4e7e-a644-2c78edd4246f None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:16.017322 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1053812c-93ec-453e-a6c6-5c1a436d801d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:16.017963 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053812c-93ec-453e-a6c6-5c1a436d801d None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:16.018245 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053812c-93ec-453e-a6c6-5c1a436d801d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:16.018518 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053812c-93ec-453e-a6c6-5c1a436d801d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:16.083583 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1053812c-93ec-453e-a6c6-5c1a436d801d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:16.085160 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1053812c-93ec-453e-a6c6-5c1a436d801d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:16.158020 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:16.181873 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 559/1144] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:56:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 16:56:16.187783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7dbb0-d1b0-4296-9471-fac79aa04e36 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:16.268991 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:16.284207 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad7dbb0-d1b0-4296-9471-fac79aa04e36 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:16.317716 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503e425b-b7c8-4a51-98ce-d32b0f7ef966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:16.318572 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e425b-b7c8-4a51-98ce-d32b0f7ef966 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:16.318858 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e425b-b7c8-4a51-98ce-d32b0f7ef966 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:16.319134 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503e425b-b7c8-4a51-98ce-d32b0f7ef966 None admin] PATH_INFO: `/v3/projects/471869c7160f4718b562793daf071c8b` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:16.320329 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503e425b-b7c8-4a51-98ce-d32b0f7ef966 None admin] RBAC: Authorizing `identity:get_project(project_id=471869c7160f4718b562793daf071c8b)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:16.321740 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503e425b-b7c8-4a51-98ce-d32b0f7ef966 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:16.323396 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 586/1145] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:56:16 2022] GET /identity/v3/projects/471869c7160f4718b562793daf071c8b => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:56:16.329296 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1053812c-93ec-453e-a6c6-5c1a436d801d None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:16.423236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1053812c-93ec-453e-a6c6-5c1a436d801d None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:16.456177 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8786792f-9536-4d54-bc67-b370513b6326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:16.456825 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786792f-9536-4d54-bc67-b370513b6326 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:16.457104 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786792f-9536-4d54-bc67-b370513b6326 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:16.457373 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786792f-9536-4d54-bc67-b370513b6326 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:16.458218 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8786792f-9536-4d54-bc67-b370513b6326 None admin] RBAC: Adding query filter params (role.id=b41f8a7a58e64f749f2099ca45cc04a7, scope.project.id=471869c7160f4718b562793daf071c8b, user.id=545bb71036bf45bfae893ac34e20a2f1) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:56:16.458521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8786792f-9536-4d54-bc67-b370513b6326 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:16.459961 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8786792f-9536-4d54-bc67-b370513b6326 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:16.465204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 560/1146] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:56:16 2022] GET /identity/v3/role_assignments?user.id=545bb71036bf45bfae893ac34e20a2f1&scope.project.id=471869c7160f4718b562793daf071c8b&role.id=b41f8a7a58e64f749f2099ca45cc04a7 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:56:18.478969 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0567024-939d-4fa2-8ae6-d5d839c3d0c3 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:18.479506 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0567024-939d-4fa2-8ae6-d5d839c3d0c3 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:18.479506 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0567024-939d-4fa2-8ae6-d5d839c3d0c3 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:18.480892 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 587/1147] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:56:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:56:18.486140 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af171d3c-ad6c-406e-aa68-44b4ec7bf8a0 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:18.486567 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af171d3c-ad6c-406e-aa68-44b4ec7bf8a0 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:18.486693 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af171d3c-ad6c-406e-aa68-44b4ec7bf8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:18.503597 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-af171d3c-ad6c-406e-aa68-44b4ec7bf8a0 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:56:18.504791 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-af171d3c-ad6c-406e-aa68-44b4ec7bf8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:56:18.533030 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:18.567651 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 561/1148] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:56:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:56:18.590165 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0567024-939d-4fa2-8ae6-d5d839c3d0c3 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:18.686718 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0567024-939d-4fa2-8ae6-d5d839c3d0c3 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:18.719060 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2a5830-0b68-410b-9ee9-68d64a095858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:18.719861 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a5830-0b68-410b-9ee9-68d64a095858 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:18.720144 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a5830-0b68-410b-9ee9-68d64a095858 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:18.720415 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2a5830-0b68-410b-9ee9-68d64a095858 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:18.731083 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2a5830-0b68-410b-9ee9-68d64a095858 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:56:18.738521 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-9d2a5830-0b68-410b-9ee9-68d64a095858 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:56:18.738521 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:56:18.738521 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:56:18.738521 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:56:18.738521 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:56:18.738521 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:56:18.738521 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:56:18.738521 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:56:18.738521 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:56:18.738521 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:56:18.738521 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:56:18.738521 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:56:18.738521 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:56:18.738521 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:56:18.738521 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:56:18.738521 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:56:18.739594 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:56:18.739594 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:56:18.739594 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:56:18.739594 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:56:18.739594 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:56:18.739594 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:56:18.739594 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:56:18.739594 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:56:18.739594 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:56:18.739594 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:56:18.739594 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:56:18.739594 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:56:18.739594 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:56:18.739594 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:56:18.739594 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:56:18.739594 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:56:18.740678 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:56:18.740678 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:56:18.740678 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:56:18.740678 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:56:18.740678 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:56:18.740678 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:56:18.740678 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:56:18.740678 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:56:18.740678 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:56:18.740678 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:56:18.740678 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:56:18.740678 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:56:18.740678 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 588/1149] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 6 16:56:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:56:18.745791 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af171d3c-ad6c-406e-aa68-44b4ec7bf8a0 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:18.840009 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af171d3c-ad6c-406e-aa68-44b4ec7bf8a0 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:18.872075 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863d5f5-4fe0-4277-bcae-cd5584ea3b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:18.872722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863d5f5-4fe0-4277-bcae-cd5584ea3b98 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:18.873001 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863d5f5-4fe0-4277-bcae-cd5584ea3b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:18.873273 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863d5f5-4fe0-4277-bcae-cd5584ea3b98 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:18.874002 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7863d5f5-4fe0-4277-bcae-cd5584ea3b98 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:56:18.874305 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7863d5f5-4fe0-4277-bcae-cd5584ea3b98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:18.875651 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7863d5f5-4fe0-4277-bcae-cd5584ea3b98 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:18.883995 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 562/1150] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:56:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:56:18.889567 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2a5830-0b68-410b-9ee9-68d64a095858 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:18.905379 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:18.987000 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2a5830-0b68-410b-9ee9-68d64a095858 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:19.019801 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32533d3e-5d01-4ec8-8434-2d7a44237c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:19.021504 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32533d3e-5d01-4ec8-8434-2d7a44237c75 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:19.021788 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32533d3e-5d01-4ec8-8434-2d7a44237c75 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:19.022086 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32533d3e-5d01-4ec8-8434-2d7a44237c75 None admin] PATH_INFO: `/v3/users/545bb71036bf45bfae893ac34e20a2f1` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:19.023358 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32533d3e-5d01-4ec8-8434-2d7a44237c75 None admin] RBAC: Authorizing `identity:get_user(user_id=545bb71036bf45bfae893ac34e20a2f1)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:19.024879 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32533d3e-5d01-4ec8-8434-2d7a44237c75 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:19.026529 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 589/1151] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:56:18 2022] GET /identity/v3/users/545bb71036bf45bfae893ac34e20a2f1 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:56:19.032483 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863d5f5-4fe0-4277-bcae-cd5584ea3b98 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:19.061770 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:19.126876 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863d5f5-4fe0-4277-bcae-cd5584ea3b98 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:19.159649 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bf8124-75f8-4094-90c4-502564c76707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:19.160512 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf8124-75f8-4094-90c4-502564c76707 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:19.160828 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf8124-75f8-4094-90c4-502564c76707 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:19.161102 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bf8124-75f8-4094-90c4-502564c76707 None admin] PATH_INFO: `/v3/projects/471869c7160f4718b562793daf071c8b` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:19.162250 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bf8124-75f8-4094-90c4-502564c76707 None admin] RBAC: Authorizing `identity:get_project(project_id=471869c7160f4718b562793daf071c8b)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:19.163648 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bf8124-75f8-4094-90c4-502564c76707 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:19.165476 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 563/1152] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:56:19 2022] GET /identity/v3/projects/471869c7160f4718b562793daf071c8b => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:56:19.171203 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32533d3e-5d01-4ec8-8434-2d7a44237c75 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:19.249423 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:19.267835 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32533d3e-5d01-4ec8-8434-2d7a44237c75 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:19.300483 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c99fc0-1c98-44d1-8695-d34d97581b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:19.301898 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c99fc0-1c98-44d1-8695-d34d97581b7d None admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:19.302183 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c99fc0-1c98-44d1-8695-d34d97581b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:19.302458 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c99fc0-1c98-44d1-8695-d34d97581b7d None admin] PATH_INFO: `/v3/projects/471869c7160f4718b562793daf071c8b/users/545bb71036bf45bfae893ac34e20a2f1/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:19.304421 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c99fc0-1c98-44d1-8695-d34d97581b7d None admin] RBAC: Authorizing `identity:create_grant(project_id=471869c7160f4718b562793daf071c8b, user_id=545bb71036bf45bfae893ac34e20a2f1, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:19.305835 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c99fc0-1c98-44d1-8695-d34d97581b7d None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:19.378805 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 590/1153] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 6 16:56:19 2022] PUT /identity/v3/projects/471869c7160f4718b562793daf071c8b/users/545bb71036bf45bfae893ac34e20a2f1/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 16:56:21.336672 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1173eb-911b-4b9c-b073-a8133ad414af None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:21.337224 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1173eb-911b-4b9c-b073-a8133ad414af None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:21.337224 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1173eb-911b-4b9c-b073-a8133ad414af None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:21.338849 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 564/1154] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:56:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:56:21.343877 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d6a337-ccde-44ab-8168-05abf0c898b4 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:21.344268 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d6a337-ccde-44ab-8168-05abf0c898b4 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:21.344448 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d6a337-ccde-44ab-8168-05abf0c898b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:21.360891 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-02d6a337-ccde-44ab-8168-05abf0c898b4 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:56:21.362074 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-02d6a337-ccde-44ab-8168-05abf0c898b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:56:21.426567 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 591/1155] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:56:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 16:56:21.449526 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1173eb-911b-4b9c-b073-a8133ad414af None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:21.543313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:21.547868 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1173eb-911b-4b9c-b073-a8133ad414af None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:21.581097 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8bd97b-2cdb-41cf-a72f-14528e9fe9c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:21.581865 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8bd97b-2cdb-41cf-a72f-14528e9fe9c8 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:21.582143 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8bd97b-2cdb-41cf-a72f-14528e9fe9c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:21.582437 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8bd97b-2cdb-41cf-a72f-14528e9fe9c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:21.592126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8bd97b-2cdb-41cf-a72f-14528e9fe9c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:56:21.599582 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-fb8bd97b-2cdb-41cf-a72f-14528e9fe9c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:56:21.599582 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:56:21.599582 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:56:21.599582 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:56:21.599582 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:56:21.599582 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:56:21.599582 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:56:21.599582 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:56:21.599582 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:56:21.599582 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:56:21.599582 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:56:21.599582 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:56:21.599582 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:56:21.599582 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:56:21.599582 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:56:21.599582 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:56:21.600780 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:56:21.600780 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:56:21.600780 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:56:21.600780 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:56:21.600780 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:56:21.600780 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:56:21.600780 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:56:21.600780 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:56:21.600780 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:56:21.600780 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:56:21.600780 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:56:21.600780 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:56:21.600780 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:56:21.600780 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:56:21.600780 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:56:21.600780 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:56:21.601972 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:56:21.601972 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:56:21.601972 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:56:21.601972 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:56:21.601972 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:56:21.601972 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:56:21.601972 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:56:21.601972 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:56:21.601972 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:56:21.601972 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:56:21.601972 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:56:21.601972 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:56:21.601972 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 565/1156] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 6 16:56:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:56:21.606560 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d6a337-ccde-44ab-8168-05abf0c898b4 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:21.673909 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:21.703923 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d6a337-ccde-44ab-8168-05abf0c898b4 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:21.736504 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46991553-1233-4028-bd8a-c35628a4ced9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:21.737176 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46991553-1233-4028-bd8a-c35628a4ced9 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:21.737458 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46991553-1233-4028-bd8a-c35628a4ced9 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:21.737734 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46991553-1233-4028-bd8a-c35628a4ced9 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:21.738483 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46991553-1233-4028-bd8a-c35628a4ced9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:56:21.739293 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46991553-1233-4028-bd8a-c35628a4ced9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:21.740748 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46991553-1233-4028-bd8a-c35628a4ced9 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:21.748586 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 592/1157] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:56:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:56:21.754826 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8bd97b-2cdb-41cf-a72f-14528e9fe9c8 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:21.850539 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8bd97b-2cdb-41cf-a72f-14528e9fe9c8 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:21.882972 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f076a3a-d44d-4622-bc58-de28aed2ebd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:21.883870 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f076a3a-d44d-4622-bc58-de28aed2ebd0 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:21.884157 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f076a3a-d44d-4622-bc58-de28aed2ebd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:21.884432 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f076a3a-d44d-4622-bc58-de28aed2ebd0 None admin] PATH_INFO: `/v3/users/545bb71036bf45bfae893ac34e20a2f1` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:21.885647 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f076a3a-d44d-4622-bc58-de28aed2ebd0 None admin] RBAC: Authorizing `identity:get_user(user_id=545bb71036bf45bfae893ac34e20a2f1)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:21.887071 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f076a3a-d44d-4622-bc58-de28aed2ebd0 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:21.888721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 566/1158] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:56:21 2022] GET /identity/v3/users/545bb71036bf45bfae893ac34e20a2f1 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:56:21.894194 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46991553-1233-4028-bd8a-c35628a4ced9 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:21.990205 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46991553-1233-4028-bd8a-c35628a4ced9 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:22.018017 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:22.023494 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adb2ef3-39c0-4810-9cb3-c072c86e4859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:22.024156 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb2ef3-39c0-4810-9cb3-c072c86e4859 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:22.024441 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb2ef3-39c0-4810-9cb3-c072c86e4859 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:22.024718 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb2ef3-39c0-4810-9cb3-c072c86e4859 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:22.088473 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adb2ef3-39c0-4810-9cb3-c072c86e4859 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:22.089900 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adb2ef3-39c0-4810-9cb3-c072c86e4859 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:22.186967 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 593/1159] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:56:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 16:56:22.193246 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f076a3a-d44d-4622-bc58-de28aed2ebd0 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:22.220201 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:22.289439 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f076a3a-d44d-4622-bc58-de28aed2ebd0 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:22.322305 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383a920c-6c7f-477d-a18a-3c74c13dc750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:22.323155 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a920c-6c7f-477d-a18a-3c74c13dc750 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:22.323436 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a920c-6c7f-477d-a18a-3c74c13dc750 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:22.323711 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383a920c-6c7f-477d-a18a-3c74c13dc750 None admin] PATH_INFO: `/v3/projects/471869c7160f4718b562793daf071c8b` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:22.324904 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383a920c-6c7f-477d-a18a-3c74c13dc750 None admin] RBAC: Authorizing `identity:get_project(project_id=471869c7160f4718b562793daf071c8b)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:22.326312 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383a920c-6c7f-477d-a18a-3c74c13dc750 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:22.328025 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 567/1160] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:56:22 2022] GET /identity/v3/projects/471869c7160f4718b562793daf071c8b => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:56:22.333903 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adb2ef3-39c0-4810-9cb3-c072c86e4859 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:22.362874 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:22.431010 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adb2ef3-39c0-4810-9cb3-c072c86e4859 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:22.464344 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828081d6-6504-4d4c-ba46-4119a13748c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:22.464936 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828081d6-6504-4d4c-ba46-4119a13748c9 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:22.465223 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828081d6-6504-4d4c-ba46-4119a13748c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:22.465503 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828081d6-6504-4d4c-ba46-4119a13748c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:22.466322 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828081d6-6504-4d4c-ba46-4119a13748c9 None admin] RBAC: Adding query filter params (role.id=b41f8a7a58e64f749f2099ca45cc04a7, scope.project.id=471869c7160f4718b562793daf071c8b, user.id=545bb71036bf45bfae893ac34e20a2f1) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:56:22.466635 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828081d6-6504-4d4c-ba46-4119a13748c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:22.468093 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828081d6-6504-4d4c-ba46-4119a13748c9 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:22.473736 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 594/1161] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:56:22 2022] GET /identity/v3/role_assignments?user.id=545bb71036bf45bfae893ac34e20a2f1&scope.project.id=471869c7160f4718b562793daf071c8b&role.id=b41f8a7a58e64f749f2099ca45cc04a7 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:56:24.477001 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10b9be-a6b0-46ea-87c1-b5bb1914acf0 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:24.477532 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10b9be-a6b0-46ea-87c1-b5bb1914acf0 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:24.477532 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d10b9be-a6b0-46ea-87c1-b5bb1914acf0 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:24.478925 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 568/1162] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:56:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:56:24.484089 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1638e150-cebe-4481-8a86-4865ea68b6c4 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:24.484486 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1638e150-cebe-4481-8a86-4865ea68b6c4 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:24.484647 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1638e150-cebe-4481-8a86-4865ea68b6c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:24.501441 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-1638e150-cebe-4481-8a86-4865ea68b6c4 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:56:24.502613 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-1638e150-cebe-4481-8a86-4865ea68b6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:56:24.566583 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 595/1163] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:56:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:56:24.588791 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d10b9be-a6b0-46ea-87c1-b5bb1914acf0 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:24.681125 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:24.688351 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d10b9be-a6b0-46ea-87c1-b5bb1914acf0 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:24.721728 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903c3877-65dd-4e31-8404-b950b887c55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:24.722372 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903c3877-65dd-4e31-8404-b950b887c55e None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:24.722656 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903c3877-65dd-4e31-8404-b950b887c55e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:24.722966 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903c3877-65dd-4e31-8404-b950b887c55e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:24.788601 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903c3877-65dd-4e31-8404-b950b887c55e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:24.790045 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903c3877-65dd-4e31-8404-b950b887c55e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:24.877179 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:24.884364 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 569/1164] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:56:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 16:56:24.890397 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1638e150-cebe-4481-8a86-4865ea68b6c4 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:24.916878 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:24.987721 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1638e150-cebe-4481-8a86-4865ea68b6c4 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:25.021762 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74076b88-8428-4fcc-bf40-c85161d98f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:25.022551 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74076b88-8428-4fcc-bf40-c85161d98f94 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:25.022835 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74076b88-8428-4fcc-bf40-c85161d98f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:25.023109 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74076b88-8428-4fcc-bf40-c85161d98f94 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:25.029651 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74076b88-8428-4fcc-bf40-c85161d98f94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:25.031085 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74076b88-8428-4fcc-bf40-c85161d98f94 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:25.045460 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-74076b88-8428-4fcc-bf40-c85161d98f94 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 06 16:56:25.045460 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:56:25.045460 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 06 16:56:25.045460 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 06 16:56:25.045460 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 16:56:25.045460 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 16:56:25.045460 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 16:56:25.045460 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 16:56:25.045460 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 06 16:56:25.045460 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:56:25.045460 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 16:56:25.045460 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:56:25.045460 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:56:25.045460 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:56:25.045460 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:56:25.045460 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:56:25.045460 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:56:25.046473 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:56:25.046473 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:56:25.046473 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:56:25.046473 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:56:25.046473 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:56:25.046473 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:56:25.046473 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:56:25.046473 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:56:25.046473 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:56:25.046473 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:56:25.046473 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:56:25.046473 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:56:25.046473 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:56:25.046473 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:56:25.046473 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:56:25.046473 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:56:25.047756 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:56:25.047756 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:56:25.047756 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:56:25.047756 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:56:25.047756 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:56:25.047756 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:56:25.047756 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:56:25.047756 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:56:25.047756 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:56:25.047756 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:56:25.047756 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 06 16:56:25.047756 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:56:25.047756 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 06 16:56:25.047756 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:56:25.047756 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 596/1165] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Jul 6 16:56:24 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:56:25.053204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903c3877-65dd-4e31-8404-b950b887c55e None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:25.147149 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903c3877-65dd-4e31-8404-b950b887c55e None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:25.179579 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44fbef5-b61c-40c1-88bb-cfa8cd11f565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:25.180248 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44fbef5-b61c-40c1-88bb-cfa8cd11f565 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:25.180529 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44fbef5-b61c-40c1-88bb-cfa8cd11f565 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:25.180800 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44fbef5-b61c-40c1-88bb-cfa8cd11f565 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:25.181506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44fbef5-b61c-40c1-88bb-cfa8cd11f565 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:56:25.181807 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44fbef5-b61c-40c1-88bb-cfa8cd11f565 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:25.183155 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44fbef5-b61c-40c1-88bb-cfa8cd11f565 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:25.190078 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 570/1166] 10.222.0.40 () {56 vars in 1115 bytes} [Wed Jul 6 16:56:25 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:56:27.176370 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca50d86-0e49-4913-a554-c5fe3073da21 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:27.176903 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca50d86-0e49-4913-a554-c5fe3073da21 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:27.176995 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca50d86-0e49-4913-a554-c5fe3073da21 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:27.178291 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 597/1167] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:56:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:56:27.183514 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c4cc9-b347-4c32-b92e-e53937ad5675 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:27.183929 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c4cc9-b347-4c32-b92e-e53937ad5675 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:27.184106 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c4cc9-b347-4c32-b92e-e53937ad5675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:27.200710 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-f18c4cc9-b347-4c32-b92e-e53937ad5675 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:56:27.201865 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-f18c4cc9-b347-4c32-b92e-e53937ad5675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:56:27.252906 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:27.266488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 571/1168] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:56:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:56:27.290621 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca50d86-0e49-4913-a554-c5fe3073da21 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:27.380250 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:27.396452 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca50d86-0e49-4913-a554-c5fe3073da21 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:27.428854 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbfa53b-3cf9-45b2-8de6-ad2645754bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:27.429444 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbfa53b-3cf9-45b2-8de6-ad2645754bf7 None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:27.429732 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbfa53b-3cf9-45b2-8de6-ad2645754bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:27.430005 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbfa53b-3cf9-45b2-8de6-ad2645754bf7 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:27.430729 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbfa53b-3cf9-45b2-8de6-ad2645754bf7 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:27.432119 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbfa53b-3cf9-45b2-8de6-ad2645754bf7 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:27.522735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 598/1169] 10.222.0.40 () {60 vars in 1138 bytes} [Wed Jul 6 16:56:27 2022] POST /identity/v3/domains => generated 267 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 16:56:29.547024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0695577-d22b-4f9a-8526-1b554707a3da None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:29.547558 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0695577-d22b-4f9a-8526-1b554707a3da None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:29.547668 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0695577-d22b-4f9a-8526-1b554707a3da None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:29.549074 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 572/1170] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:56:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:56:29.554108 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7bea1-c8eb-4a02-a419-ec5cb018312e None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:29.554612 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7bea1-c8eb-4a02-a419-ec5cb018312e None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:29.554706 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7bea1-c8eb-4a02-a419-ec5cb018312e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:29.571484 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-3ba7bea1-c8eb-4a02-a419-ec5cb018312e None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:56:29.572695 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-3ba7bea1-c8eb-4a02-a419-ec5cb018312e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:56:29.636065 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 599/1171] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:56:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:56:29.659449 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0695577-d22b-4f9a-8526-1b554707a3da None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:29.759098 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0695577-d22b-4f9a-8526-1b554707a3da None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:29.791303 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b34427-840f-4fd6-bd20-21e43218867a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:29.792179 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b34427-840f-4fd6-bd20-21e43218867a None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:29.792465 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b34427-840f-4fd6-bd20-21e43218867a None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:29.792739 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b34427-840f-4fd6-bd20-21e43218867a None admin] PATH_INFO: `/v3/domains/9b0bdeee0de64528a42b2e6ed0a88e41` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:29.803267 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b34427-840f-4fd6-bd20-21e43218867a None admin] RBAC: Authorizing `identity:get_domain(domain_id=9b0bdeee0de64528a42b2e6ed0a88e41)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:29.804676 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b34427-840f-4fd6-bd20-21e43218867a None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:29.806289 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 573/1172] 10.222.0.40 () {56 vars in 1183 bytes} [Wed Jul 6 16:56:29 2022] GET /identity/v3/domains/9b0bdeee0de64528a42b2e6ed0a88e41 => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:56:29.812122 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba7bea1-c8eb-4a02-a419-ec5cb018312e None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:29.885922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:29.907467 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba7bea1-c8eb-4a02-a419-ec5cb018312e None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:29.939963 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8501be51-07de-441f-a54f-f8f6473bd4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:29.943355 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501be51-07de-441f-a54f-f8f6473bd4bc None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:29.943355 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501be51-07de-441f-a54f-f8f6473bd4bc None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:29.943355 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8501be51-07de-441f-a54f-f8f6473bd4bc None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:29.943355 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8501be51-07de-441f-a54f-f8f6473bd4bc None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:29.943874 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8501be51-07de-441f-a54f-f8f6473bd4bc None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:30.077469 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 600/1173] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Jul 6 16:56:29 2022] POST /identity/v3/projects => generated 361 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 16:56:32.114690 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ba9aa-b4bb-440f-8f9e-6855a0e57242 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:32.115191 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ba9aa-b4bb-440f-8f9e-6855a0e57242 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:32.115191 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0ba9aa-b4bb-440f-8f9e-6855a0e57242 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:32.116585 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 574/1174] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:56:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:56:32.121794 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b37e0-b939-45a7-8360-9553d3f38aca None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:32.122177 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b37e0-b939-45a7-8360-9553d3f38aca None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:32.122354 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b37e0-b939-45a7-8360-9553d3f38aca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:32.138676 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-014b37e0-b939-45a7-8360-9553d3f38aca None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:56:32.139881 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-014b37e0-b939-45a7-8360-9553d3f38aca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:56:32.203263 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 601/1175] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:56:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:56:32.231589 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ba9aa-b4bb-440f-8f9e-6855a0e57242 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:32.280628 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:32.330845 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0ba9aa-b4bb-440f-8f9e-6855a0e57242 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:32.362975 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3437ee80-137c-4d0f-8267-d1def2e25575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:32.363854 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437ee80-137c-4d0f-8267-d1def2e25575 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:32.364145 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437ee80-137c-4d0f-8267-d1def2e25575 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:32.364469 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3437ee80-137c-4d0f-8267-d1def2e25575 None admin] PATH_INFO: `/v3/domains/9b0bdeee0de64528a42b2e6ed0a88e41` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:32.365756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3437ee80-137c-4d0f-8267-d1def2e25575 None admin] RBAC: Authorizing `identity:get_domain(domain_id=9b0bdeee0de64528a42b2e6ed0a88e41)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:32.367112 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3437ee80-137c-4d0f-8267-d1def2e25575 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:32.368766 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 575/1176] 10.222.0.40 () {56 vars in 1183 bytes} [Wed Jul 6 16:56:32 2022] GET /identity/v3/domains/9b0bdeee0de64528a42b2e6ed0a88e41 => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:56:32.374765 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b37e0-b939-45a7-8360-9553d3f38aca None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:32.432048 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:32.476631 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b37e0-b939-45a7-8360-9553d3f38aca None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:32.510287 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e45c1c9-743d-494c-a77f-b19c09ddbc67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:32.510972 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e45c1c9-743d-494c-a77f-b19c09ddbc67 None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:32.511263 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e45c1c9-743d-494c-a77f-b19c09ddbc67 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:32.511540 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e45c1c9-743d-494c-a77f-b19c09ddbc67 None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:32.512278 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e45c1c9-743d-494c-a77f-b19c09ddbc67 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:32.513706 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e45c1c9-743d-494c-a77f-b19c09ddbc67 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:32.603579 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 602/1177] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 6 16:56:32 2022] POST /identity/v3/users => generated 311 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 16:56:34.650488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462fde84-a655-4833-830f-c7c65f79efb3 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:34.651122 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462fde84-a655-4833-830f-c7c65f79efb3 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:34.651122 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462fde84-a655-4833-830f-c7c65f79efb3 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:34.652434 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 576/1178] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:56:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:56:34.657590 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4516d63-38a2-418f-bc53-13836920d01e None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:34.658025 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4516d63-38a2-418f-bc53-13836920d01e None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:34.658128 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4516d63-38a2-418f-bc53-13836920d01e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:34.674923 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-f4516d63-38a2-418f-bc53-13836920d01e None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:56:34.676124 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f4516d63-38a2-418f-bc53-13836920d01e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:56:34.740643 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 603/1179] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:56:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 16:56:34.763305 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462fde84-a655-4833-830f-c7c65f79efb3 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:34.861116 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462fde84-a655-4833-830f-c7c65f79efb3 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:34.877163 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:34.893762 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec204304-db9e-4bde-8d40-fea7c57e60db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:34.894540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec204304-db9e-4bde-8d40-fea7c57e60db None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:34.894823 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec204304-db9e-4bde-8d40-fea7c57e60db None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:34.895097 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec204304-db9e-4bde-8d40-fea7c57e60db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:34.905102 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec204304-db9e-4bde-8d40-fea7c57e60db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:56:34.913554 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-ec204304-db9e-4bde-8d40-fea7c57e60db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:56:34.913554 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:56:34.913554 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:56:34.913554 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:56:34.913554 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:56:34.913554 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:56:34.913554 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:56:34.913554 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:56:34.913554 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:56:34.913554 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:56:34.913554 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:56:34.913554 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:56:34.913554 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:56:34.913554 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:56:34.913554 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:56:34.913554 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:56:34.914711 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:56:34.914711 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:56:34.914711 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:56:34.914711 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:56:34.914711 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:56:34.914711 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:56:34.914711 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:56:34.914711 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:56:34.914711 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:56:34.914711 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:56:34.914711 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:56:34.914711 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:56:34.914711 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:56:34.914711 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:56:34.914711 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:56:34.914711 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:56:34.915802 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:56:34.915802 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:56:34.915802 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:56:34.915802 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:56:34.915802 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:56:34.915802 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:56:34.915802 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:56:34.915802 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:56:34.915802 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:56:34.915802 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:56:34.915802 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:56:34.915802 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:56:34.915802 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 577/1180] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 6 16:56:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:56:34.920967 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4516d63-38a2-418f-bc53-13836920d01e None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:34.965207 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:35.016414 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4516d63-38a2-418f-bc53-13836920d01e None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:35.049758 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25491b00-0d39-405b-a237-82e33e3b083c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:35.050416 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25491b00-0d39-405b-a237-82e33e3b083c None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:35.050700 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25491b00-0d39-405b-a237-82e33e3b083c None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:35.050978 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25491b00-0d39-405b-a237-82e33e3b083c None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:35.051721 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25491b00-0d39-405b-a237-82e33e3b083c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:56:35.052055 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25491b00-0d39-405b-a237-82e33e3b083c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:35.053423 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25491b00-0d39-405b-a237-82e33e3b083c None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:35.061219 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 604/1181] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:56:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:56:35.066747 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec204304-db9e-4bde-8d40-fea7c57e60db None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:35.163718 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec204304-db9e-4bde-8d40-fea7c57e60db None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:35.195394 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581fbc07-e752-4818-96cc-50718300d7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:35.196286 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581fbc07-e752-4818-96cc-50718300d7d3 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:35.196570 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581fbc07-e752-4818-96cc-50718300d7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:35.196843 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581fbc07-e752-4818-96cc-50718300d7d3 None admin] PATH_INFO: `/v3/users/ba921ff739a74ceb8194ccc2a7f35c64` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:35.209726 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581fbc07-e752-4818-96cc-50718300d7d3 None admin] RBAC: Authorizing `identity:get_user(user_id=ba921ff739a74ceb8194ccc2a7f35c64)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:35.211149 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581fbc07-e752-4818-96cc-50718300d7d3 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:35.212775 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 578/1182] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:56:35 2022] GET /identity/v3/users/ba921ff739a74ceb8194ccc2a7f35c64 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:56:35.218167 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25491b00-0d39-405b-a237-82e33e3b083c None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:35.315708 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25491b00-0d39-405b-a237-82e33e3b083c None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:35.322998 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:35.349443 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74228e37-6a47-47d8-a672-7a1e905e687f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:35.350083 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74228e37-6a47-47d8-a672-7a1e905e687f None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:35.350372 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74228e37-6a47-47d8-a672-7a1e905e687f None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:35.350668 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74228e37-6a47-47d8-a672-7a1e905e687f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:35.416659 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74228e37-6a47-47d8-a672-7a1e905e687f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:35.418056 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74228e37-6a47-47d8-a672-7a1e905e687f None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:35.513107 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 605/1183] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:56:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 16:56:35.518884 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581fbc07-e752-4818-96cc-50718300d7d3 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:35.551852 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:35.613438 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581fbc07-e752-4818-96cc-50718300d7d3 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:35.645543 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15439096-bc02-46b3-aa8a-87e1829ae376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:35.646931 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15439096-bc02-46b3-aa8a-87e1829ae376 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:35.647209 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15439096-bc02-46b3-aa8a-87e1829ae376 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:35.647478 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15439096-bc02-46b3-aa8a-87e1829ae376 None admin] PATH_INFO: `/v3/projects/4a25bea621014cb3917add56b9c25e1c` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:35.648661 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15439096-bc02-46b3-aa8a-87e1829ae376 None admin] RBAC: Authorizing `identity:get_project(project_id=4a25bea621014cb3917add56b9c25e1c)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:35.650072 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15439096-bc02-46b3-aa8a-87e1829ae376 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:35.651749 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 579/1184] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:56:35 2022] GET /identity/v3/projects/4a25bea621014cb3917add56b9c25e1c => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:56:35.657808 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74228e37-6a47-47d8-a672-7a1e905e687f None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:35.665890 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:35.754781 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74228e37-6a47-47d8-a672-7a1e905e687f None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:35.786965 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7159b7a-cc65-4a6c-87bd-a1d9dac17918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:35.787605 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7159b7a-cc65-4a6c-87bd-a1d9dac17918 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:35.787918 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7159b7a-cc65-4a6c-87bd-a1d9dac17918 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:35.788195 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7159b7a-cc65-4a6c-87bd-a1d9dac17918 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:35.789013 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7159b7a-cc65-4a6c-87bd-a1d9dac17918 None admin] RBAC: Adding query filter params (role.id=b41f8a7a58e64f749f2099ca45cc04a7, scope.project.id=4a25bea621014cb3917add56b9c25e1c, user.id=ba921ff739a74ceb8194ccc2a7f35c64) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:56:35.789321 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7159b7a-cc65-4a6c-87bd-a1d9dac17918 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:35.790741 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7159b7a-cc65-4a6c-87bd-a1d9dac17918 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:35.795985 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 606/1185] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:56:35 2022] GET /identity/v3/role_assignments?user.id=ba921ff739a74ceb8194ccc2a7f35c64&scope.project.id=4a25bea621014cb3917add56b9c25e1c&role.id=b41f8a7a58e64f749f2099ca45cc04a7 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:56:37.758302 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a2c5b-3fa6-4b75-a299-a5c14f2c2c49 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:37.758739 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a2c5b-3fa6-4b75-a299-a5c14f2c2c49 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:37.758846 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0a2c5b-3fa6-4b75-a299-a5c14f2c2c49 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:37.760274 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 580/1186] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:56:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:56:37.765504 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174c289-ddca-42be-9a39-1aca4fac86c3 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:37.765899 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174c289-ddca-42be-9a39-1aca4fac86c3 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:37.766062 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174c289-ddca-42be-9a39-1aca4fac86c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:37.782709 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-6174c289-ddca-42be-9a39-1aca4fac86c3 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:56:37.783916 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-6174c289-ddca-42be-9a39-1aca4fac86c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:56:37.844125 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:37.848520 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 607/1187] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:56:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:56:37.871766 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0a2c5b-3fa6-4b75-a299-a5c14f2c2c49 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:37.967948 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0a2c5b-3fa6-4b75-a299-a5c14f2c2c49 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:37.969411 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:38.000422 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e56baed-93a5-4710-b9b1-0e62d189e6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:38.001127 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e56baed-93a5-4710-b9b1-0e62d189e6ad None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:38.001407 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e56baed-93a5-4710-b9b1-0e62d189e6ad None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:38.001678 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e56baed-93a5-4710-b9b1-0e62d189e6ad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:38.011708 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e56baed-93a5-4710-b9b1-0e62d189e6ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:56:38.019239 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-8e56baed-93a5-4710-b9b1-0e62d189e6ad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:56:38.019239 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:56:38.019239 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:56:38.019239 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:56:38.019239 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:56:38.019239 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:56:38.019239 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:56:38.019239 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:56:38.019239 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:56:38.019239 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:56:38.019239 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:56:38.019239 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:56:38.019239 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:56:38.019239 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:56:38.019239 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:56:38.019239 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:56:38.020350 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:56:38.020350 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:56:38.020350 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:56:38.020350 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:56:38.020350 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:56:38.020350 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:56:38.020350 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:56:38.020350 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:56:38.020350 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:56:38.020350 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:56:38.020350 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:56:38.020350 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:56:38.020350 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:56:38.020350 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:56:38.020350 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:56:38.020350 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:56:38.022225 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:56:38.022225 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:56:38.022225 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:56:38.022225 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:56:38.022225 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:56:38.022225 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:56:38.022225 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:56:38.022225 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:56:38.022225 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:56:38.022225 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:56:38.022225 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:56:38.022225 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:56:38.022225 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 581/1188] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 6 16:56:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:56:38.027139 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6174c289-ddca-42be-9a39-1aca4fac86c3 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:38.121951 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6174c289-ddca-42be-9a39-1aca4fac86c3 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:38.154773 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b4ed0-49a9-4e02-9df3-dbb3ae92ed71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:38.155423 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b4ed0-49a9-4e02-9df3-dbb3ae92ed71 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:38.155707 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b4ed0-49a9-4e02-9df3-dbb3ae92ed71 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:38.156015 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b4ed0-49a9-4e02-9df3-dbb3ae92ed71 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:38.156750 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8b4ed0-49a9-4e02-9df3-dbb3ae92ed71 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:56:38.157055 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8b4ed0-49a9-4e02-9df3-dbb3ae92ed71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:38.158412 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8b4ed0-49a9-4e02-9df3-dbb3ae92ed71 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:38.166738 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 608/1189] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:56:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:56:38.172170 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e56baed-93a5-4710-b9b1-0e62d189e6ad None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:38.267398 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e56baed-93a5-4710-b9b1-0e62d189e6ad None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:38.300688 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a376230-2716-4158-b122-ba193bae5139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:38.301589 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a376230-2716-4158-b122-ba193bae5139 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:38.301954 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a376230-2716-4158-b122-ba193bae5139 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:38.302349 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a376230-2716-4158-b122-ba193bae5139 None admin] PATH_INFO: `/v3/users/ba921ff739a74ceb8194ccc2a7f35c64` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:38.303551 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a376230-2716-4158-b122-ba193bae5139 None admin] RBAC: Authorizing `identity:get_user(user_id=ba921ff739a74ceb8194ccc2a7f35c64)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:38.305001 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a376230-2716-4158-b122-ba193bae5139 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:38.306646 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 582/1190] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:56:38 2022] GET /identity/v3/users/ba921ff739a74ceb8194ccc2a7f35c64 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:56:38.312615 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b4ed0-49a9-4e02-9df3-dbb3ae92ed71 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:38.373450 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:38.411900 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b4ed0-49a9-4e02-9df3-dbb3ae92ed71 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:38.445503 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcee3e1-88ae-4bc9-88ab-7215616c6709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:38.446539 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcee3e1-88ae-4bc9-88ab-7215616c6709 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:38.446828 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcee3e1-88ae-4bc9-88ab-7215616c6709 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:38.447127 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcee3e1-88ae-4bc9-88ab-7215616c6709 None admin] PATH_INFO: `/v3/projects/4a25bea621014cb3917add56b9c25e1c` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:38.448697 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcee3e1-88ae-4bc9-88ab-7215616c6709 None admin] RBAC: Authorizing `identity:get_project(project_id=4a25bea621014cb3917add56b9c25e1c)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:38.450866 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcee3e1-88ae-4bc9-88ab-7215616c6709 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:38.452688 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 609/1191] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:56:38 2022] GET /identity/v3/projects/4a25bea621014cb3917add56b9c25e1c => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:56:38.460570 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a376230-2716-4158-b122-ba193bae5139 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:38.490231 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:38.556843 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a376230-2716-4158-b122-ba193bae5139 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:38.590007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e89735f-bed2-4f3f-ac1e-e4e281dccc92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:38.590887 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89735f-bed2-4f3f-ac1e-e4e281dccc92 None admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:38.591170 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89735f-bed2-4f3f-ac1e-e4e281dccc92 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:38.591446 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e89735f-bed2-4f3f-ac1e-e4e281dccc92 None admin] PATH_INFO: `/v3/projects/4a25bea621014cb3917add56b9c25e1c/users/ba921ff739a74ceb8194ccc2a7f35c64/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:38.593383 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e89735f-bed2-4f3f-ac1e-e4e281dccc92 None admin] RBAC: Authorizing `identity:create_grant(project_id=4a25bea621014cb3917add56b9c25e1c, user_id=ba921ff739a74ceb8194ccc2a7f35c64, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:38.594800 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e89735f-bed2-4f3f-ac1e-e4e281dccc92 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:38.694955 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 583/1192] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 6 16:56:38 2022] PUT /identity/v3/projects/4a25bea621014cb3917add56b9c25e1c/users/ba921ff739a74ceb8194ccc2a7f35c64/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 16:56:40.665572 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14132076-5b59-4c89-8ed2-fce579511222 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:40.666129 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14132076-5b59-4c89-8ed2-fce579511222 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:40.666129 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14132076-5b59-4c89-8ed2-fce579511222 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:40.667480 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 610/1193] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:56:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:56:40.672624 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d08b5-c607-4157-acb5-1523407a16c0 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:40.673004 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d08b5-c607-4157-acb5-1523407a16c0 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:40.673184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720d08b5-c607-4157-acb5-1523407a16c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:40.690018 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-720d08b5-c607-4157-acb5-1523407a16c0 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:56:40.691176 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-720d08b5-c607-4157-acb5-1523407a16c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:56:40.757760 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 584/1194] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:56:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:56:40.782854 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14132076-5b59-4c89-8ed2-fce579511222 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:40.882263 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14132076-5b59-4c89-8ed2-fce579511222 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:40.909519 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:40.915631 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b99772-64ca-4e93-829c-e90876b14a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:40.916397 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b99772-64ca-4e93-829c-e90876b14a7b None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:40.916724 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b99772-64ca-4e93-829c-e90876b14a7b None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:40.917003 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b99772-64ca-4e93-829c-e90876b14a7b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:40.927385 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b99772-64ca-4e93-829c-e90876b14a7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:56:40.935094 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-d3b99772-64ca-4e93-829c-e90876b14a7b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:56:40.935094 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:56:40.935094 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:56:40.935094 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:56:40.935094 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:56:40.935094 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:56:40.935094 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:56:40.935094 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:56:40.935094 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:56:40.935094 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:56:40.935094 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:56:40.935094 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:56:40.935094 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:56:40.935094 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:56:40.935094 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:56:40.935094 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:56:40.936236 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:56:40.936236 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:56:40.936236 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:56:40.936236 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:56:40.936236 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:56:40.936236 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:56:40.936236 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:56:40.936236 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:56:40.936236 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:56:40.936236 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:56:40.936236 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:56:40.936236 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:56:40.936236 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:56:40.936236 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:56:40.936236 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:56:40.936236 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:56:40.937236 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:56:40.937236 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:56:40.937236 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:56:40.937236 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:56:40.937236 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:56:40.937236 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:56:40.937236 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:56:40.937236 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:56:40.937236 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:56:40.937236 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:56:40.937236 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:56:40.937236 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:56:40.937236 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 611/1195] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 6 16:56:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:56:40.942322 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d08b5-c607-4157-acb5-1523407a16c0 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:40.963192 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:41.039333 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720d08b5-c607-4157-acb5-1523407a16c0 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:41.071211 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f534b-9f85-4a63-8da8-488300639051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:41.071918 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f534b-9f85-4a63-8da8-488300639051 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:41.072207 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f534b-9f85-4a63-8da8-488300639051 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:41.072487 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369f534b-9f85-4a63-8da8-488300639051 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:41.073237 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369f534b-9f85-4a63-8da8-488300639051 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:56:41.073545 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369f534b-9f85-4a63-8da8-488300639051 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:41.074903 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369f534b-9f85-4a63-8da8-488300639051 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:41.082738 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 585/1196] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:56:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:56:41.088882 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b99772-64ca-4e93-829c-e90876b14a7b None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:41.187156 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b99772-64ca-4e93-829c-e90876b14a7b None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:41.219798 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92464b8-2749-41c2-8013-90f7f8c2c0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:41.220613 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92464b8-2749-41c2-8013-90f7f8c2c0ad None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:41.220898 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92464b8-2749-41c2-8013-90f7f8c2c0ad None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:41.221174 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92464b8-2749-41c2-8013-90f7f8c2c0ad None admin] PATH_INFO: `/v3/users/ba921ff739a74ceb8194ccc2a7f35c64` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:41.222340 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92464b8-2749-41c2-8013-90f7f8c2c0ad None admin] RBAC: Authorizing `identity:get_user(user_id=ba921ff739a74ceb8194ccc2a7f35c64)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:41.224352 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92464b8-2749-41c2-8013-90f7f8c2c0ad None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:41.225935 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 612/1197] 10.222.0.40 () {56 vars in 1177 bytes} [Wed Jul 6 16:56:41 2022] GET /identity/v3/users/ba921ff739a74ceb8194ccc2a7f35c64 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:56:41.231394 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f534b-9f85-4a63-8da8-488300639051 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:41.309660 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:41.326969 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369f534b-9f85-4a63-8da8-488300639051 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:41.358978 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99024909-3bed-464d-be4b-c499f48d5dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:41.359611 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99024909-3bed-464d-be4b-c499f48d5dff None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:41.359930 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99024909-3bed-464d-be4b-c499f48d5dff None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:41.360206 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99024909-3bed-464d-be4b-c499f48d5dff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:41.423328 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99024909-3bed-464d-be4b-c499f48d5dff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:41.424783 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99024909-3bed-464d-be4b-c499f48d5dff None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:41.498958 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:41.517074 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 586/1198] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:56:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 16:56:41.523650 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92464b8-2749-41c2-8013-90f7f8c2c0ad None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:41.578665 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:41.617524 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92464b8-2749-41c2-8013-90f7f8c2c0ad None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:41.650825 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45ecfb2-377a-4cae-88db-b194405bde25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:41.651677 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ecfb2-377a-4cae-88db-b194405bde25 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:41.651990 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ecfb2-377a-4cae-88db-b194405bde25 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:41.652265 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45ecfb2-377a-4cae-88db-b194405bde25 None admin] PATH_INFO: `/v3/projects/4a25bea621014cb3917add56b9c25e1c` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:41.653431 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45ecfb2-377a-4cae-88db-b194405bde25 None admin] RBAC: Authorizing `identity:get_project(project_id=4a25bea621014cb3917add56b9c25e1c)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:41.654876 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45ecfb2-377a-4cae-88db-b194405bde25 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:41.656581 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 613/1199] 10.222.0.40 () {56 vars in 1186 bytes} [Wed Jul 6 16:56:41 2022] GET /identity/v3/projects/4a25bea621014cb3917add56b9c25e1c => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:56:41.662957 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99024909-3bed-464d-be4b-c499f48d5dff None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:56:41.757431 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99024909-3bed-464d-be4b-c499f48d5dff None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:56:41.790635 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ed9182-42a1-4edb-917f-d55e748b6d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:56:41.791280 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed9182-42a1-4edb-917f-d55e748b6d77 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:56:41.791562 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed9182-42a1-4edb-917f-d55e748b6d77 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:56:41.791865 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ed9182-42a1-4edb-917f-d55e748b6d77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:56:41.792681 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ed9182-42a1-4edb-917f-d55e748b6d77 None admin] RBAC: Adding query filter params (role.id=b41f8a7a58e64f749f2099ca45cc04a7, scope.project.id=4a25bea621014cb3917add56b9c25e1c, user.id=ba921ff739a74ceb8194ccc2a7f35c64) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:56:41.792989 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ed9182-42a1-4edb-917f-d55e748b6d77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:56:41.794409 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ed9182-42a1-4edb-917f-d55e748b6d77 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:56:41.800236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 587/1200] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:56:41 2022] GET /identity/v3/role_assignments?user.id=ba921ff739a74ceb8194ccc2a7f35c64&scope.project.id=4a25bea621014cb3917add56b9c25e1c&role.id=b41f8a7a58e64f749f2099ca45cc04a7 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:56:51.645584 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=37 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:56:51.797562 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=77 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:57:54.565615 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ff768-a4ba-4306-9925-f4e049825639 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:57:54.566204 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ff768-a4ba-4306-9925-f4e049825639 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:57:54.566204 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419ff768-a4ba-4306-9925-f4e049825639 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:57:54.567529 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 614/1201] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:57:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:57:54.573502 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f3d47-e341-4ef6-a330-073e8b6166ba None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:57:54.577798 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f3d47-e341-4ef6-a330-073e8b6166ba None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:57:54.577798 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f3d47-e341-4ef6-a330-073e8b6166ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:57:54.591109 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-5b7f3d47-e341-4ef6-a330-073e8b6166ba None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:57:54.592313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-5b7f3d47-e341-4ef6-a330-073e8b6166ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:57:54.656845 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 588/1202] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:57:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:57:54.681415 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419ff768-a4ba-4306-9925-f4e049825639 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:57:54.782978 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419ff768-a4ba-4306-9925-f4e049825639 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:57:54.816453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8805574b-91dc-4f86-9210-0153265d23bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:57:54.817174 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8805574b-91dc-4f86-9210-0153265d23bd None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:57:54.817466 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8805574b-91dc-4f86-9210-0153265d23bd None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:57:54.817739 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8805574b-91dc-4f86-9210-0153265d23bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:57:54.827716 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8805574b-91dc-4f86-9210-0153265d23bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:57:54.829182 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8805574b-91dc-4f86-9210-0153265d23bd None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:57:54.846783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-8805574b-91dc-4f86-9210-0153265d23bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:57:54.846783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:57:54.846783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:57:54.846783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:57:54.846783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:57:54.846783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:57:54.846783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:57:54.846783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:57:54.846783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:57:54.846783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:57:54.846783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:57:54.846783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:57:54.846783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:57:54.846783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:57:54.846783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:57:54.847919 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:57:54.847919 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:57:54.847919 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:57:54.847919 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:57:54.847919 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:57:54.847919 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:57:54.847919 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:57:54.847919 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:57:54.847919 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:57:54.847919 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:57:54.847919 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:57:54.847919 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:57:54.847919 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:57:54.847919 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:57:54.847919 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:57:54.847919 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:57:54.847919 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:57:54.848952 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:57:54.848952 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:57:54.848952 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:57:54.848952 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:57:54.848952 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 615/1203] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:57:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:57:54.854680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7f3d47-e341-4ef6-a330-073e8b6166ba None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:57:54.951073 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7f3d47-e341-4ef6-a330-073e8b6166ba None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:57:54.979549 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:57:54.985238 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f54a6-1177-4f24-98cc-badf29142528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:57:54.985899 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f54a6-1177-4f24-98cc-badf29142528 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:57:54.986178 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f54a6-1177-4f24-98cc-badf29142528 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:57:54.986450 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f54a6-1177-4f24-98cc-badf29142528 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:57:54.987165 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f54a6-1177-4f24-98cc-badf29142528 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:57:54.987468 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f54a6-1177-4f24-98cc-badf29142528 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:57:54.988912 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f54a6-1177-4f24-98cc-badf29142528 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:57:55.000144 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 589/1204] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:57:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:57:55.005935 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8805574b-91dc-4f86-9210-0153265d23bd None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:57:55.058778 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:57:55.103646 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8805574b-91dc-4f86-9210-0153265d23bd None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:57:55.134890 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deca7c47-1d43-4fd3-a692-22d8fe62d0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:57:55.135553 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca7c47-1d43-4fd3-a692-22d8fe62d0ff None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:57:55.135862 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca7c47-1d43-4fd3-a692-22d8fe62d0ff None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:57:55.136138 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deca7c47-1d43-4fd3-a692-22d8fe62d0ff None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:57:55.136824 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deca7c47-1d43-4fd3-a692-22d8fe62d0ff None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:57:55.138244 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deca7c47-1d43-4fd3-a692-22d8fe62d0ff None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:57:55.215350 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 616/1205] 10.222.0.40 () {60 vars in 1132 bytes} [Wed Jul 6 16:57:55 2022] POST /identity/v3/users => generated 251 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 16:57:57.253165 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bba442-76fc-459d-bd78-5d5036f3ff7b None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:57:57.253696 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bba442-76fc-459d-bd78-5d5036f3ff7b None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:57:57.253793 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bba442-76fc-459d-bd78-5d5036f3ff7b None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:57:57.255089 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 590/1206] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:57:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:57:57.260845 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9134bc1-92e2-4fe0-9738-8cb686605ce9 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:57:57.261229 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9134bc1-92e2-4fe0-9738-8cb686605ce9 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:57:57.261405 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9134bc1-92e2-4fe0-9738-8cb686605ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:57:57.280947 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-e9134bc1-92e2-4fe0-9738-8cb686605ce9 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:57:57.282127 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-e9134bc1-92e2-4fe0-9738-8cb686605ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:57:57.347969 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 617/1207] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:57:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:57:57.370978 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bba442-76fc-459d-bd78-5d5036f3ff7b None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:57:57.471118 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bba442-76fc-459d-bd78-5d5036f3ff7b None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:57:57.503771 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17141458-2f48-405f-b591-78571e22b2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:57:57.504508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17141458-2f48-405f-b591-78571e22b2e0 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:57:57.504789 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17141458-2f48-405f-b591-78571e22b2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:57:57.505058 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17141458-2f48-405f-b591-78571e22b2e0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:57:57.515544 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17141458-2f48-405f-b591-78571e22b2e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:57:57.523855 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-17141458-2f48-405f-b591-78571e22b2e0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 16:57:57.523855 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:57:57.523855 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:57:57.523855 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:57:57.523855 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:57:57.523855 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:57:57.523855 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:57:57.523855 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:57:57.523855 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:57:57.523855 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:57:57.523855 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:57:57.523855 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:57:57.523855 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:57:57.523855 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:57:57.523855 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:57:57.523855 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:57:57.524949 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:57:57.524949 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:57:57.524949 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:57:57.524949 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:57:57.524949 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:57:57.524949 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:57:57.524949 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:57:57.524949 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:57:57.524949 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:57:57.524949 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:57:57.524949 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:57:57.524949 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:57:57.524949 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:57:57.524949 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:57:57.524949 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:57:57.524949 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:57:57.525984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:57:57.525984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:57:57.525984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:57:57.525984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:57:57.525984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:57:57.525984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:57:57.525984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:57:57.525984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:57:57.525984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:57:57.525984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:57:57.525984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 16:57:57.525984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:57:57.525984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 591/1208] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 6 16:57:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:57:57.531047 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9134bc1-92e2-4fe0-9738-8cb686605ce9 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:57:57.570620 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:57:57.626919 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9134bc1-92e2-4fe0-9738-8cb686605ce9 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:57:57.660585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff83f6c-5678-47d9-98f8-3c3dcf5e2803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:57:57.661240 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff83f6c-5678-47d9-98f8-3c3dcf5e2803 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:57:57.661522 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff83f6c-5678-47d9-98f8-3c3dcf5e2803 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:57:57.661799 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff83f6c-5678-47d9-98f8-3c3dcf5e2803 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:57:57.662560 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff83f6c-5678-47d9-98f8-3c3dcf5e2803 None admin] RBAC: Adding query filter params (name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:57:57.662867 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff83f6c-5678-47d9-98f8-3c3dcf5e2803 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:57:57.664287 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff83f6c-5678-47d9-98f8-3c3dcf5e2803 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:57:57.672004 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 618/1209] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 6 16:57:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:57:57.678263 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17141458-2f48-405f-b591-78571e22b2e0 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:57:57.728878 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:57:57.778321 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17141458-2f48-405f-b591-78571e22b2e0 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:57:57.813462 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e3f5a-aa61-4f63-a771-f0d8a8330a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:57:57.814245 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e3f5a-aa61-4f63-a771-f0d8a8330a56 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:57:57.814534 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e3f5a-aa61-4f63-a771-f0d8a8330a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:57:57.814808 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2e3f5a-aa61-4f63-a771-f0d8a8330a56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:57:57.824972 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2e3f5a-aa61-4f63-a771-f0d8a8330a56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:57:57.826405 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2e3f5a-aa61-4f63-a771-f0d8a8330a56 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:57:57.843165 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-8d2e3f5a-aa61-4f63-a771-f0d8a8330a56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:57:57.843165 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:57:57.843165 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:57:57.843165 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:57:57.843165 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:57:57.843165 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:57:57.843165 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:57:57.843165 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:57:57.843165 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:57:57.843165 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:57:57.843165 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:57:57.843165 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:57:57.843165 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:57:57.843165 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:57:57.843165 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:57:57.846825 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:57:57.846825 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:57:57.846825 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:57:57.846825 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:57:57.846825 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:57:57.846825 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:57:57.846825 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:57:57.846825 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:57:57.846825 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:57:57.846825 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:57:57.846825 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:57:57.846825 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:57:57.846825 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:57:57.846825 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:57:57.846825 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:57:57.846825 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:57:57.846825 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:57:57.847937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:57:57.847937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:57:57.847937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:57:57.847937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:57:57.847937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 592/1210] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:57:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:57:57.850764 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff83f6c-5678-47d9-98f8-3c3dcf5e2803 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:57:57.946121 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:57:57.949285 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff83f6c-5678-47d9-98f8-3c3dcf5e2803 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:57:57.982350 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d367a9-2d22-4a9d-97a4-404fe851d695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:57:57.983049 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d367a9-2d22-4a9d-97a4-404fe851d695 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:57:57.983342 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d367a9-2d22-4a9d-97a4-404fe851d695 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:57:57.983618 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d367a9-2d22-4a9d-97a4-404fe851d695 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:57:57.984376 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d367a9-2d22-4a9d-97a4-404fe851d695 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:57:57.984684 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d367a9-2d22-4a9d-97a4-404fe851d695 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:57:57.986085 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d367a9-2d22-4a9d-97a4-404fe851d695 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:57:57.999100 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 619/1211] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:57:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:57:58.004737 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e3f5a-aa61-4f63-a771-f0d8a8330a56 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:57:58.101880 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2e3f5a-aa61-4f63-a771-f0d8a8330a56 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:57:58.103304 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:57:58.135217 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d1b9cc-df65-4ee9-96ea-da216d21a2ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:57:58.136039 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d1b9cc-df65-4ee9-96ea-da216d21a2ce None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:57:58.136320 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d1b9cc-df65-4ee9-96ea-da216d21a2ce None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:57:58.136589 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d1b9cc-df65-4ee9-96ea-da216d21a2ce None admin] PATH_INFO: `/v3/users/placement` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:57:58.143234 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d1b9cc-df65-4ee9-96ea-da216d21a2ce None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:57:58.144700 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d1b9cc-df65-4ee9-96ea-da216d21a2ce None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:57:58.159506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-12d1b9cc-df65-4ee9-96ea-da216d21a2ce None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 06 16:57:58.159506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:57:58.159506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:57:58.159506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:57:58.159506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:57:58.159506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:57:58.159506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:57:58.159506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:57:58.159506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:57:58.159506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:57:58.159506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:57:58.159506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:57:58.159506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 16:57:58.159506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 16:57:58.159506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 16:57:58.159506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 16:57:58.160540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:57:58.160540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:57:58.160540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 16:57:58.160540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:57:58.160540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 16:57:58.160540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:57:58.160540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:57:58.160540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:57:58.160540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:57:58.160540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:57:58.160540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:57:58.160540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:57:58.160540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:57:58.160540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:57:58.160540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:57:58.160540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:57:58.160540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:57:58.160540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:57:58.161725 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:57:58.161725 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:57:58.161725 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 16:57:58.161725 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 16:57:58.161725 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 16:57:58.161725 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 16:57:58.161725 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 16:57:58.161725 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 16:57:58.161725 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 06 16:57:58.161725 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:57:58.161725 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 593/1212] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:57:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:57:58.166984 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d367a9-2d22-4a9d-97a4-404fe851d695 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:57:58.264655 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d367a9-2d22-4a9d-97a4-404fe851d695 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:57:58.298148 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d1ba8b-3a93-4323-934a-f59fa068141b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:57:58.298806 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1ba8b-3a93-4323-934a-f59fa068141b None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:57:58.299088 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1ba8b-3a93-4323-934a-f59fa068141b None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:57:58.299361 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1ba8b-3a93-4323-934a-f59fa068141b None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:57:58.300210 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d1ba8b-3a93-4323-934a-f59fa068141b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:57:58.300515 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d1ba8b-3a93-4323-934a-f59fa068141b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:57:58.301917 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d1ba8b-3a93-4323-934a-f59fa068141b None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:57:58.318672 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 620/1213] 10.222.0.40 () {56 vars in 1143 bytes} [Wed Jul 6 16:57:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:57:58.324377 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d1b9cc-df65-4ee9-96ea-da216d21a2ce None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:57:58.422477 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d1b9cc-df65-4ee9-96ea-da216d21a2ce None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:57:58.455601 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189a9cf-b0b9-4380-8f61-33da91361a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:57:58.456277 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189a9cf-b0b9-4380-8f61-33da91361a27 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:57:58.456559 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189a9cf-b0b9-4380-8f61-33da91361a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:57:58.456836 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5189a9cf-b0b9-4380-8f61-33da91361a27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:57:58.485489 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:57:58.523288 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5189a9cf-b0b9-4380-8f61-33da91361a27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:57:58.524862 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5189a9cf-b0b9-4380-8f61-33da91361a27 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:57:58.622588 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 594/1214] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:57:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 16:57:58.628433 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d1ba8b-3a93-4323-934a-f59fa068141b None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:57:58.648514 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:57:58.727341 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d1ba8b-3a93-4323-934a-f59fa068141b None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:57:58.762193 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bada38-a72e-4c9c-87b1-a34c7507d3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:57:58.762985 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bada38-a72e-4c9c-87b1-a34c7507d3d4 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:57:58.763268 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bada38-a72e-4c9c-87b1-a34c7507d3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:57:58.763542 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bada38-a72e-4c9c-87b1-a34c7507d3d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:57:58.774324 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bada38-a72e-4c9c-87b1-a34c7507d3d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:57:58.775782 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bada38-a72e-4c9c-87b1-a34c7507d3d4 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:57:58.793474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-70bada38-a72e-4c9c-87b1-a34c7507d3d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:57:58.793474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:57:58.793474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:57:58.793474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:57:58.793474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:57:58.793474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:57:58.793474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:57:58.793474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:57:58.793474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:57:58.793474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:57:58.793474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:57:58.793474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:57:58.793474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:57:58.793474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:57:58.793474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:57:58.794538 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:57:58.794538 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:57:58.794538 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:57:58.794538 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:57:58.794538 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:57:58.794538 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:57:58.794538 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:57:58.794538 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:57:58.794538 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:57:58.794538 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:57:58.794538 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:57:58.794538 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:57:58.794538 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:57:58.794538 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:57:58.794538 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:57:58.794538 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:57:58.794538 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:57:58.795639 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:57:58.795639 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:57:58.795639 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:57:58.795639 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:57:58.795639 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 621/1215] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:57:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:57:58.800542 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189a9cf-b0b9-4380-8f61-33da91361a27 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:57:58.859533 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:57:58.897314 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5189a9cf-b0b9-4380-8f61-33da91361a27 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:57:58.929786 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5592cd68-d361-42b2-854b-17104734b65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:57:58.930441 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592cd68-d361-42b2-854b-17104734b65f None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:57:58.930727 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592cd68-d361-42b2-854b-17104734b65f None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:57:58.931001 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5592cd68-d361-42b2-854b-17104734b65f None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:57:58.931784 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5592cd68-d361-42b2-854b-17104734b65f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:57:58.932090 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5592cd68-d361-42b2-854b-17104734b65f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:57:58.933447 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5592cd68-d361-42b2-854b-17104734b65f None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:57:58.945314 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 595/1216] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:57:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:57:58.950980 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bada38-a72e-4c9c-87b1-a34c7507d3d4 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:57:59.019109 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:57:59.049662 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bada38-a72e-4c9c-87b1-a34c7507d3d4 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:57:59.083508 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f68e4-f184-49d9-95ad-8137fba0dbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:57:59.084334 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f68e4-f184-49d9-95ad-8137fba0dbf1 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:57:59.084620 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f68e4-f184-49d9-95ad-8137fba0dbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:57:59.084895 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f68e4-f184-49d9-95ad-8137fba0dbf1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:57:59.091555 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f68e4-f184-49d9-95ad-8137fba0dbf1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:57:59.093066 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f68e4-f184-49d9-95ad-8137fba0dbf1 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:57:59.107929 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-0f3f68e4-f184-49d9-95ad-8137fba0dbf1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:57:59.107929 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:57:59.107929 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:57:59.107929 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:57:59.107929 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:57:59.107929 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:57:59.107929 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:57:59.107929 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:57:59.107929 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:57:59.107929 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:57:59.107929 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:57:59.107929 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:57:59.107929 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 16:57:59.107929 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 16:57:59.107929 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 16:57:59.109006 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 16:57:59.109006 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:57:59.109006 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:57:59.109006 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:57:59.109006 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:57:59.109006 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:57:59.109006 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:57:59.109006 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:57:59.109006 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:57:59.109006 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:57:59.109006 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:57:59.109006 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:57:59.109006 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:57:59.109006 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:57:59.109006 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:57:59.109006 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:57:59.109006 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:57:59.114791 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 16:57:59.114791 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 16:57:59.114791 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 16:57:59.114791 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 16:57:59.114791 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 16:57:59.114791 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 16:57:59.114791 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:57:59.114791 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:57:59.114791 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 622/1217] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:57:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:57:59.115889 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5592cd68-d361-42b2-854b-17104734b65f None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:57:59.215738 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5592cd68-d361-42b2-854b-17104734b65f None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:57:59.237662 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:57:59.248881 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76eba96a-4726-4101-a15d-e39801e5a3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:57:59.249530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eba96a-4726-4101-a15d-e39801e5a3af None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:57:59.249811 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eba96a-4726-4101-a15d-e39801e5a3af None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:57:59.250084 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76eba96a-4726-4101-a15d-e39801e5a3af None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:57:59.250887 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eba96a-4726-4101-a15d-e39801e5a3af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:57:59.251192 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eba96a-4726-4101-a15d-e39801e5a3af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:57:59.252794 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76eba96a-4726-4101-a15d-e39801e5a3af None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:57:59.264445 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 596/1218] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 6 16:57:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:57:59.275266 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f68e4-f184-49d9-95ad-8137fba0dbf1 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:57:59.367737 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f68e4-f184-49d9-95ad-8137fba0dbf1 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:57:59.396117 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:57:59.403489 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d15b05-e45a-4413-9af5-bb5c421c0901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:57:59.407888 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d15b05-e45a-4413-9af5-bb5c421c0901 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:57:59.407888 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d15b05-e45a-4413-9af5-bb5c421c0901 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:57:59.407888 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d15b05-e45a-4413-9af5-bb5c421c0901 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:57:59.407888 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d15b05-e45a-4413-9af5-bb5c421c0901 None admin] RBAC: Adding query filter params (role.id=750bca9c32ac4aa49adcbee30463f1fa, scope.project.id=7938b0346d4240ab89774327d09a89ae, user.id=76ca877bee9041d28ad41e45ef3dd18c) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:57:59.407888 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d15b05-e45a-4413-9af5-bb5c421c0901 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:57:59.407888 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d15b05-e45a-4413-9af5-bb5c421c0901 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:57:59.412556 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 623/1219] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:57:59 2022] GET /identity/v3/role_assignments?user.id=76ca877bee9041d28ad41e45ef3dd18c&scope.project.id=7938b0346d4240ab89774327d09a89ae&role.id=750bca9c32ac4aa49adcbee30463f1fa => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:58:01.412774 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae42d87-55e3-4061-b366-0694c97ab058 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:01.413325 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae42d87-55e3-4061-b366-0694c97ab058 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:01.413325 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae42d87-55e3-4061-b366-0694c97ab058 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:01.414694 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 597/1220] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:58:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:58:01.420052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88543c7d-8cd0-460f-8111-e445d2ace382 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:01.420473 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88543c7d-8cd0-460f-8111-e445d2ace382 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:01.420628 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88543c7d-8cd0-460f-8111-e445d2ace382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:01.441664 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-88543c7d-8cd0-460f-8111-e445d2ace382 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:58:01.442880 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-88543c7d-8cd0-460f-8111-e445d2ace382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:58:01.508656 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 624/1221] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:58:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 06 16:58:01.531017 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae42d87-55e3-4061-b366-0694c97ab058 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:01.631591 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae42d87-55e3-4061-b366-0694c97ab058 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:01.671408 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa346e0a-ab29-49da-bde2-a434bf8f929b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:01.672151 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa346e0a-ab29-49da-bde2-a434bf8f929b None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:01.672437 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa346e0a-ab29-49da-bde2-a434bf8f929b None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:01.672711 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa346e0a-ab29-49da-bde2-a434bf8f929b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:01.684257 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa346e0a-ab29-49da-bde2-a434bf8f929b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:58:01.692119 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-fa346e0a-ab29-49da-bde2-a434bf8f929b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 16:58:01.692119 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:58:01.692119 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:58:01.692119 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:58:01.692119 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:58:01.692119 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:58:01.692119 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:58:01.692119 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:58:01.692119 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:58:01.692119 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:58:01.692119 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:58:01.692119 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:58:01.692119 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:58:01.692119 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:58:01.692119 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:58:01.692119 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:58:01.694662 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:58:01.694662 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:58:01.694662 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:58:01.694662 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:58:01.694662 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:58:01.694662 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:58:01.694662 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:58:01.694662 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:58:01.694662 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:58:01.694662 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:58:01.694662 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:58:01.694662 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:58:01.694662 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:58:01.694662 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:58:01.694662 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:58:01.694662 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:58:01.695679 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:58:01.695679 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:58:01.695679 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:58:01.695679 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:58:01.695679 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:58:01.695679 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:58:01.695679 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:58:01.695679 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:58:01.695679 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:58:01.695679 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:58:01.695679 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 16:58:01.695679 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:58:01.695679 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 598/1222] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 6 16:58:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:58:01.699588 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88543c7d-8cd0-460f-8111-e445d2ace382 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:01.796938 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88543c7d-8cd0-460f-8111-e445d2ace382 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:01.819634 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:01.829714 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934eae06-9a45-460a-bea9-2cdd7634d49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:01.830375 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934eae06-9a45-460a-bea9-2cdd7634d49c None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:01.830658 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934eae06-9a45-460a-bea9-2cdd7634d49c None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:01.830932 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934eae06-9a45-460a-bea9-2cdd7634d49c None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:01.831669 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934eae06-9a45-460a-bea9-2cdd7634d49c None admin] RBAC: Adding query filter params (name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:58:01.831994 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934eae06-9a45-460a-bea9-2cdd7634d49c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:01.833343 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934eae06-9a45-460a-bea9-2cdd7634d49c None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:01.841402 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 625/1223] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 6 16:58:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:58:01.847088 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa346e0a-ab29-49da-bde2-a434bf8f929b None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:01.889138 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:01.942544 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa346e0a-ab29-49da-bde2-a434bf8f929b None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:01.975281 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4994440c-d192-45e8-84d7-b808d445ffb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:01.976106 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994440c-d192-45e8-84d7-b808d445ffb7 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:01.976393 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994440c-d192-45e8-84d7-b808d445ffb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:01.976699 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4994440c-d192-45e8-84d7-b808d445ffb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:01.987070 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4994440c-d192-45e8-84d7-b808d445ffb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:01.988556 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4994440c-d192-45e8-84d7-b808d445ffb7 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:02.005594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-4994440c-d192-45e8-84d7-b808d445ffb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:58:02.005594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:58:02.005594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:58:02.005594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:58:02.005594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:58:02.005594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:58:02.005594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:58:02.005594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:58:02.005594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:58:02.005594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:58:02.005594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:58:02.005594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:58:02.005594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:58:02.005594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:58:02.005594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:58:02.006783 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:58:02.006783 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:58:02.006783 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:58:02.006783 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:58:02.006783 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:58:02.006783 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:58:02.006783 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:58:02.006783 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:58:02.006783 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:58:02.006783 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:58:02.006783 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:58:02.006783 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:58:02.006783 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:58:02.006783 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:58:02.006783 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:58:02.006783 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:58:02.006783 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:58:02.007847 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:58:02.007847 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:58:02.007847 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:58:02.007847 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:58:02.007847 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 599/1224] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:58:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:58:02.012923 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934eae06-9a45-460a-bea9-2cdd7634d49c None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:02.113267 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934eae06-9a45-460a-bea9-2cdd7634d49c None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:02.145949 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc8278-bf19-4cb5-8522-440b990dd110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:02.147105 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc8278-bf19-4cb5-8522-440b990dd110 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:02.147385 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc8278-bf19-4cb5-8522-440b990dd110 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:02.147662 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dc8278-bf19-4cb5-8522-440b990dd110 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:02.148400 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dc8278-bf19-4cb5-8522-440b990dd110 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:58:02.148702 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dc8278-bf19-4cb5-8522-440b990dd110 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:02.150051 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dc8278-bf19-4cb5-8522-440b990dd110 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:02.161479 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 626/1225] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:58:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:58:02.167744 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4994440c-d192-45e8-84d7-b808d445ffb7 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:02.259443 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:02.265928 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4994440c-d192-45e8-84d7-b808d445ffb7 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:02.299214 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d3e931-853c-4221-a0c1-cbb318f7e3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:02.300057 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3e931-853c-4221-a0c1-cbb318f7e3ea None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:02.300344 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3e931-853c-4221-a0c1-cbb318f7e3ea None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:02.300620 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3e931-853c-4221-a0c1-cbb318f7e3ea None admin] PATH_INFO: `/v3/users/placement` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:02.307798 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d3e931-853c-4221-a0c1-cbb318f7e3ea None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:02.309228 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d3e931-853c-4221-a0c1-cbb318f7e3ea None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:02.324253 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-71d3e931-853c-4221-a0c1-cbb318f7e3ea None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 06 16:58:02.324253 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:58:02.324253 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:58:02.324253 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:58:02.324253 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:58:02.324253 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:58:02.324253 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:58:02.324253 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:58:02.324253 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:58:02.324253 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:58:02.324253 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:58:02.324253 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:58:02.324253 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 16:58:02.324253 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 16:58:02.324253 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 16:58:02.324253 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 16:58:02.325206 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:58:02.325206 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:58:02.325206 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 16:58:02.325206 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:58:02.325206 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 16:58:02.325206 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:58:02.325206 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:58:02.325206 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:58:02.325206 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:58:02.325206 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:58:02.325206 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:58:02.325206 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:58:02.325206 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:58:02.325206 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:58:02.325206 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:58:02.325206 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:58:02.325206 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:58:02.325206 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:58:02.326363 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:58:02.326363 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:58:02.326363 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 16:58:02.326363 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 16:58:02.326363 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 16:58:02.326363 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 16:58:02.326363 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 16:58:02.326363 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 16:58:02.326363 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 06 16:58:02.326363 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:58:02.326363 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 600/1226] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:58:02 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:58:02.331346 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc8278-bf19-4cb5-8522-440b990dd110 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:02.377694 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:02.428685 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dc8278-bf19-4cb5-8522-440b990dd110 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:02.461764 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18109fa-c63e-4a20-ab8f-3ffcf20bee90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:02.462437 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18109fa-c63e-4a20-ab8f-3ffcf20bee90 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:02.462724 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18109fa-c63e-4a20-ab8f-3ffcf20bee90 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:02.463004 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18109fa-c63e-4a20-ab8f-3ffcf20bee90 None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:02.463879 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18109fa-c63e-4a20-ab8f-3ffcf20bee90 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:58:02.464193 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18109fa-c63e-4a20-ab8f-3ffcf20bee90 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:02.465627 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18109fa-c63e-4a20-ab8f-3ffcf20bee90 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:02.483012 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 627/1227] 10.222.0.40 () {56 vars in 1143 bytes} [Wed Jul 6 16:58:02 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:58:02.488777 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d3e931-853c-4221-a0c1-cbb318f7e3ea None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:02.590452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d3e931-853c-4221-a0c1-cbb318f7e3ea None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:02.624147 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c66c8d-0083-4df5-acfe-aa85348c621e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:02.624934 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c66c8d-0083-4df5-acfe-aa85348c621e None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:02.625219 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c66c8d-0083-4df5-acfe-aa85348c621e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:02.625498 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c66c8d-0083-4df5-acfe-aa85348c621e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:02.635710 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c66c8d-0083-4df5-acfe-aa85348c621e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:02.637191 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c66c8d-0083-4df5-acfe-aa85348c621e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:02.655383 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-16c66c8d-0083-4df5-acfe-aa85348c621e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:58:02.655383 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:58:02.655383 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:58:02.655383 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:58:02.655383 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:58:02.655383 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:58:02.655383 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:58:02.655383 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:58:02.655383 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:58:02.655383 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:58:02.655383 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:58:02.655383 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:58:02.655383 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:58:02.655383 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:58:02.655383 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:58:02.656343 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:58:02.656343 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:58:02.656343 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:58:02.656343 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:58:02.656343 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:58:02.656343 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:58:02.656343 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:58:02.656343 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:58:02.656343 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:58:02.656343 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:58:02.656343 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:58:02.656343 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:58:02.656343 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:58:02.656343 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:58:02.656343 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:58:02.656343 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:58:02.656343 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:58:02.657269 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:58:02.657269 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:58:02.657269 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:58:02.657269 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:58:02.657269 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 601/1228] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:58:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 06 16:58:02.662431 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18109fa-c63e-4a20-ab8f-3ffcf20bee90 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:02.761470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18109fa-c63e-4a20-ab8f-3ffcf20bee90 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:02.763855 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:02.795230 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912cd367-08db-4586-82b4-793da8d4b3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:02.795934 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912cd367-08db-4586-82b4-793da8d4b3fa None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:02.796222 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912cd367-08db-4586-82b4-793da8d4b3fa None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:02.796501 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912cd367-08db-4586-82b4-793da8d4b3fa None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:02.797221 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912cd367-08db-4586-82b4-793da8d4b3fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:58:02.797531 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912cd367-08db-4586-82b4-793da8d4b3fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:02.798894 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912cd367-08db-4586-82b4-793da8d4b3fa None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:02.810522 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 628/1229] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:58:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:58:02.816273 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c66c8d-0083-4df5-acfe-aa85348c621e None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:02.826641 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:02.913997 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c66c8d-0083-4df5-acfe-aa85348c621e None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:02.947365 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b8009-b8e1-4b95-ab4d-2abaa95b04c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:02.948176 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b8009-b8e1-4b95-ab4d-2abaa95b04c0 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:02.948488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b8009-b8e1-4b95-ab4d-2abaa95b04c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:02.948783 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0b8009-b8e1-4b95-ab4d-2abaa95b04c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:02.955359 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b8009-b8e1-4b95-ab4d-2abaa95b04c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:02.956825 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0b8009-b8e1-4b95-ab4d-2abaa95b04c0 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:02.972543 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-ce0b8009-b8e1-4b95-ab4d-2abaa95b04c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:58:02.972543 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:58:02.972543 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:58:02.972543 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:58:02.972543 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:58:02.972543 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:58:02.972543 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:58:02.972543 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:58:02.972543 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:58:02.972543 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:58:02.972543 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:58:02.972543 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:58:02.972543 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 16:58:02.972543 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 16:58:02.972543 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 16:58:02.976419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 16:58:02.976419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:58:02.976419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:58:02.976419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:58:02.976419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:58:02.976419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:58:02.976419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:58:02.976419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:58:02.976419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:58:02.976419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:58:02.976419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:58:02.976419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:58:02.976419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:58:02.976419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:58:02.976419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:58:02.976419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:58:02.976419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:58:02.977532 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 16:58:02.977532 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 16:58:02.977532 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 16:58:02.977532 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 16:58:02.977532 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 16:58:02.977532 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 16:58:02.977532 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:58:02.977532 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:58:02.977532 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 602/1230] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:58:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:58:02.980073 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912cd367-08db-4586-82b4-793da8d4b3fa None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:03.077672 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912cd367-08db-4586-82b4-793da8d4b3fa None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:03.109811 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2d754f-b78f-421e-93d3-4f5fda5d617d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:03.110462 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d754f-b78f-421e-93d3-4f5fda5d617d None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:03.110743 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d754f-b78f-421e-93d3-4f5fda5d617d None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:03.111018 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2d754f-b78f-421e-93d3-4f5fda5d617d None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:03.111814 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2d754f-b78f-421e-93d3-4f5fda5d617d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:58:03.112122 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2d754f-b78f-421e-93d3-4f5fda5d617d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:03.113539 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2d754f-b78f-421e-93d3-4f5fda5d617d None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:03.125742 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 629/1231] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 6 16:58:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:58:03.131766 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b8009-b8e1-4b95-ab4d-2abaa95b04c0 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:03.197165 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:03.230647 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0b8009-b8e1-4b95-ab4d-2abaa95b04c0 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:03.263240 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d52818-d9f1-4cd2-8bbb-3fe16416b30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:03.264162 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d52818-d9f1-4cd2-8bbb-3fe16416b30e None admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:03.264445 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d52818-d9f1-4cd2-8bbb-3fe16416b30e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:03.264717 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d52818-d9f1-4cd2-8bbb-3fe16416b30e None admin] PATH_INFO: `/v3/projects/7938b0346d4240ab89774327d09a89ae/users/76ca877bee9041d28ad41e45ef3dd18c/roles/750bca9c32ac4aa49adcbee30463f1fa` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:03.278492 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d52818-d9f1-4cd2-8bbb-3fe16416b30e None admin] RBAC: Authorizing `identity:create_grant(project_id=7938b0346d4240ab89774327d09a89ae, user_id=76ca877bee9041d28ad41e45ef3dd18c, role_id=750bca9c32ac4aa49adcbee30463f1fa)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:03.279954 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d52818-d9f1-4cd2-8bbb-3fe16416b30e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:03.356338 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 603/1232] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 6 16:58:03 2022] PUT /identity/v3/projects/7938b0346d4240ab89774327d09a89ae/users/76ca877bee9041d28ad41e45ef3dd18c/roles/750bca9c32ac4aa49adcbee30463f1fa => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 16:58:05.414542 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d2b9f-399b-42c6-a201-32c9970fc944 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:05.415080 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d2b9f-399b-42c6-a201-32c9970fc944 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:05.415200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4d2b9f-399b-42c6-a201-32c9970fc944 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:05.416542 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 630/1233] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:58:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:58:05.421587 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc104252-6a4a-44e7-a586-50cea82e05a9 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:05.422213 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc104252-6a4a-44e7-a586-50cea82e05a9 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:05.422213 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc104252-6a4a-44e7-a586-50cea82e05a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:05.439204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-bc104252-6a4a-44e7-a586-50cea82e05a9 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:58:05.440437 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-bc104252-6a4a-44e7-a586-50cea82e05a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:58:05.518422 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 604/1234] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:58:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:58:05.545683 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d2b9f-399b-42c6-a201-32c9970fc944 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:05.571574 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:05.655953 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4d2b9f-399b-42c6-a201-32c9970fc944 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:05.705253 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22678ac7-f35f-4913-858c-52424f5cdad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:05.705974 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22678ac7-f35f-4913-858c-52424f5cdad6 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:05.706262 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22678ac7-f35f-4913-858c-52424f5cdad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:05.706539 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22678ac7-f35f-4913-858c-52424f5cdad6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:05.716972 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22678ac7-f35f-4913-858c-52424f5cdad6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:58:05.726554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-22678ac7-f35f-4913-858c-52424f5cdad6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 06 16:58:05.726554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:58:05.726554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:58:05.726554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:58:05.726554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:58:05.726554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:58:05.726554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:58:05.726554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:58:05.726554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:58:05.726554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:58:05.726554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:58:05.726554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:58:05.726554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:58:05.726554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:58:05.726554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:58:05.726554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:58:05.727672 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:58:05.727672 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:58:05.727672 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:58:05.727672 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:58:05.727672 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:58:05.727672 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:58:05.727672 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:58:05.727672 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:58:05.727672 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:58:05.727672 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:58:05.727672 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:58:05.727672 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:58:05.727672 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:58:05.727672 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:58:05.727672 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:58:05.727672 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:58:05.728891 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:58:05.728891 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:58:05.728891 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:58:05.728891 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:58:05.728891 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:58:05.728891 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:58:05.728891 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:58:05.728891 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:58:05.728891 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:58:05.728891 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:58:05.728891 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 06 16:58:05.728891 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:58:05.728891 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 631/1235] 10.222.0.40 () {56 vars in 1102 bytes} [Wed Jul 6 16:58:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:58:05.735303 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc104252-6a4a-44e7-a586-50cea82e05a9 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:05.783500 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:05.835426 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc104252-6a4a-44e7-a586-50cea82e05a9 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:05.868529 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e3c8e6-c689-44a5-b87f-76e550d347c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:05.869203 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e3c8e6-c689-44a5-b87f-76e550d347c8 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:05.869523 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e3c8e6-c689-44a5-b87f-76e550d347c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:05.869806 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e3c8e6-c689-44a5-b87f-76e550d347c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:05.870552 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e3c8e6-c689-44a5-b87f-76e550d347c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:58:05.870867 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e3c8e6-c689-44a5-b87f-76e550d347c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:05.872322 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e3c8e6-c689-44a5-b87f-76e550d347c8 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:05.880093 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 605/1236] 10.222.0.40 () {56 vars in 1103 bytes} [Wed Jul 6 16:58:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:58:05.886386 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22678ac7-f35f-4913-858c-52424f5cdad6 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:05.969595 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:05.986205 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22678ac7-f35f-4913-858c-52424f5cdad6 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:06.019100 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dd688c-94d1-45fb-a427-c919f8638cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:06.019913 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd688c-94d1-45fb-a427-c919f8638cdc None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:06.020201 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd688c-94d1-45fb-a427-c919f8638cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:06.020479 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd688c-94d1-45fb-a427-c919f8638cdc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:06.030512 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dd688c-94d1-45fb-a427-c919f8638cdc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:06.031970 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dd688c-94d1-45fb-a427-c919f8638cdc None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:06.049805 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-52dd688c-94d1-45fb-a427-c919f8638cdc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:58:06.049805 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:58:06.049805 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:58:06.049805 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:58:06.049805 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:58:06.049805 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:58:06.049805 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:58:06.049805 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:58:06.049805 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:58:06.049805 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:58:06.049805 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:58:06.049805 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:58:06.049805 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:58:06.049805 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:58:06.049805 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:58:06.051657 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:58:06.051657 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:58:06.051657 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:58:06.051657 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:58:06.051657 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:58:06.051657 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:58:06.051657 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:58:06.051657 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:58:06.051657 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:58:06.051657 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:58:06.051657 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:58:06.051657 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:58:06.051657 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:58:06.051657 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:58:06.051657 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:58:06.051657 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:58:06.051657 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:58:06.053157 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:58:06.053157 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:58:06.053157 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:58:06.053157 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:58:06.053157 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 632/1237] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:58:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:58:06.057310 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e3c8e6-c689-44a5-b87f-76e550d347c8 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:06.156371 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e3c8e6-c689-44a5-b87f-76e550d347c8 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:06.165552 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:06.190199 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ba541-f8f8-4bbd-86fd-89e72354beed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:06.190864 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ba541-f8f8-4bbd-86fd-89e72354beed None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:06.191149 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ba541-f8f8-4bbd-86fd-89e72354beed None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:06.191424 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23ba541-f8f8-4bbd-86fd-89e72354beed None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:06.192178 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23ba541-f8f8-4bbd-86fd-89e72354beed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:58:06.192486 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23ba541-f8f8-4bbd-86fd-89e72354beed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:06.193888 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23ba541-f8f8-4bbd-86fd-89e72354beed None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:06.205551 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 606/1238] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:58:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:58:06.211247 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dd688c-94d1-45fb-a427-c919f8638cdc None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:06.314070 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dd688c-94d1-45fb-a427-c919f8638cdc None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:06.346894 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92b4aa-95c6-44e3-b1f6-58241bf9c599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:06.347693 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92b4aa-95c6-44e3-b1f6-58241bf9c599 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:06.348011 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92b4aa-95c6-44e3-b1f6-58241bf9c599 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:06.348289 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92b4aa-95c6-44e3-b1f6-58241bf9c599 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:06.351517 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:06.355989 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf92b4aa-95c6-44e3-b1f6-58241bf9c599 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:06.362033 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf92b4aa-95c6-44e3-b1f6-58241bf9c599 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:06.371979 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-bf92b4aa-95c6-44e3-b1f6-58241bf9c599 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 06 16:58:06.371979 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:58:06.371979 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:58:06.371979 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:58:06.371979 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:58:06.371979 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:58:06.371979 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:58:06.371979 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:58:06.371979 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:58:06.371979 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:58:06.371979 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:58:06.371979 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:58:06.371979 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 16:58:06.371979 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 16:58:06.371979 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 16:58:06.371979 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 16:58:06.373103 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:58:06.373103 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:58:06.373103 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 16:58:06.373103 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:58:06.373103 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 16:58:06.373103 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:58:06.373103 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:58:06.373103 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:58:06.373103 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:58:06.373103 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:58:06.373103 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:58:06.373103 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:58:06.373103 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:58:06.373103 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:58:06.373103 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:58:06.373103 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:58:06.373103 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:58:06.373103 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:58:06.374475 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:58:06.374475 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:58:06.374475 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 16:58:06.374475 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 16:58:06.374475 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 16:58:06.374475 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 16:58:06.374475 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 16:58:06.374475 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 16:58:06.374475 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 06 16:58:06.374475 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:58:06.374475 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 633/1239] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:58:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:58:06.379012 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ba541-f8f8-4bbd-86fd-89e72354beed None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:06.477392 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23ba541-f8f8-4bbd-86fd-89e72354beed None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:06.509621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b9c73-b6a0-45d0-b63b-675d59a9b4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:06.510283 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b9c73-b6a0-45d0-b63b-675d59a9b4b4 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:06.510568 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b9c73-b6a0-45d0-b63b-675d59a9b4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:06.510844 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b9c73-b6a0-45d0-b63b-675d59a9b4b4 None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:06.511670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b9c73-b6a0-45d0-b63b-675d59a9b4b4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:58:06.512032 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b9c73-b6a0-45d0-b63b-675d59a9b4b4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:06.513506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30b9c73-b6a0-45d0-b63b-675d59a9b4b4 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:06.530886 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 607/1240] 10.222.0.40 () {56 vars in 1143 bytes} [Wed Jul 6 16:58:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:58:06.537589 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92b4aa-95c6-44e3-b1f6-58241bf9c599 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:06.642081 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92b4aa-95c6-44e3-b1f6-58241bf9c599 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:06.675930 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228d6f53-43c4-47a1-9abc-887b41a0bbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:06.676575 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d6f53-43c4-47a1-9abc-887b41a0bbe5 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:06.676859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d6f53-43c4-47a1-9abc-887b41a0bbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:06.677135 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228d6f53-43c4-47a1-9abc-887b41a0bbe5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:06.737103 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:06.743524 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228d6f53-43c4-47a1-9abc-887b41a0bbe5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:06.752815 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228d6f53-43c4-47a1-9abc-887b41a0bbe5 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:06.844399 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 634/1241] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:58:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 06 16:58:06.850458 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b9c73-b6a0-45d0-b63b-675d59a9b4b4 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:06.877220 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:06.948218 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b9c73-b6a0-45d0-b63b-675d59a9b4b4 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:06.980971 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07fa670-f294-4962-a721-1c195c27cfb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:06.981765 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07fa670-f294-4962-a721-1c195c27cfb4 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:06.982047 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07fa670-f294-4962-a721-1c195c27cfb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:06.982324 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07fa670-f294-4962-a721-1c195c27cfb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:06.992159 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07fa670-f294-4962-a721-1c195c27cfb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:06.995814 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07fa670-f294-4962-a721-1c195c27cfb4 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:07.011226 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-b07fa670-f294-4962-a721-1c195c27cfb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:58:07.011226 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:58:07.011226 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:58:07.011226 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:58:07.011226 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:58:07.011226 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:58:07.011226 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:58:07.011226 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:58:07.011226 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:58:07.011226 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:58:07.011226 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:58:07.011226 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:58:07.011226 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:58:07.011226 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:58:07.011226 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:58:07.012382 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:58:07.012382 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:58:07.012382 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:58:07.012382 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:58:07.012382 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:58:07.012382 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:58:07.012382 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:58:07.012382 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:58:07.012382 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:58:07.012382 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:58:07.012382 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:58:07.012382 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:58:07.012382 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:58:07.012382 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:58:07.012382 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:58:07.012382 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:58:07.012382 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:58:07.013548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:58:07.013548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:58:07.013548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:58:07.013548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:58:07.013548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 608/1242] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:58:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:58:07.018508 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228d6f53-43c4-47a1-9abc-887b41a0bbe5 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:07.109781 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:07.115873 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228d6f53-43c4-47a1-9abc-887b41a0bbe5 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:07.149285 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03934ca7-370f-4ab4-9190-138c8e921811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:07.149948 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03934ca7-370f-4ab4-9190-138c8e921811 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:07.150230 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03934ca7-370f-4ab4-9190-138c8e921811 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:07.150503 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03934ca7-370f-4ab4-9190-138c8e921811 None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:07.151218 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03934ca7-370f-4ab4-9190-138c8e921811 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:58:07.151522 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03934ca7-370f-4ab4-9190-138c8e921811 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:07.153056 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03934ca7-370f-4ab4-9190-138c8e921811 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:07.164553 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 635/1243] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:58:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:58:07.170262 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07fa670-f294-4962-a721-1c195c27cfb4 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:07.247515 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:07.268380 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07fa670-f294-4962-a721-1c195c27cfb4 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:07.302305 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea340b-73f7-42bc-b0a0-ac8907aa59a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:07.303625 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea340b-73f7-42bc-b0a0-ac8907aa59a6 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:07.303936 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea340b-73f7-42bc-b0a0-ac8907aa59a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:07.304215 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea340b-73f7-42bc-b0a0-ac8907aa59a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:07.310797 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea340b-73f7-42bc-b0a0-ac8907aa59a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:07.312282 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea340b-73f7-42bc-b0a0-ac8907aa59a6 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:07.326427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-f3ea340b-73f7-42bc-b0a0-ac8907aa59a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:58:07.326427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:58:07.326427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:58:07.326427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:58:07.326427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:58:07.326427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:58:07.326427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:58:07.326427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:58:07.326427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:58:07.326427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:58:07.326427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:58:07.326427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:58:07.326427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 16:58:07.326427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 16:58:07.326427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 16:58:07.330701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 16:58:07.330701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:58:07.330701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:58:07.330701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:58:07.330701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:58:07.330701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:58:07.330701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:58:07.330701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:58:07.330701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:58:07.330701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:58:07.330701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:58:07.330701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:58:07.330701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:58:07.330701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:58:07.330701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:58:07.330701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:58:07.330701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:58:07.331857 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 16:58:07.331857 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 16:58:07.331857 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 16:58:07.331857 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 16:58:07.331857 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 16:58:07.331857 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 16:58:07.331857 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:58:07.331857 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:58:07.331857 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 609/1244] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:58:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:58:07.333785 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03934ca7-370f-4ab4-9190-138c8e921811 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:07.432386 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03934ca7-370f-4ab4-9190-138c8e921811 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:07.465572 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d26e341-e2c3-41be-8d72-69cbf5944555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:07.466232 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d26e341-e2c3-41be-8d72-69cbf5944555 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:07.466539 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d26e341-e2c3-41be-8d72-69cbf5944555 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:07.466846 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d26e341-e2c3-41be-8d72-69cbf5944555 None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:07.467628 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d26e341-e2c3-41be-8d72-69cbf5944555 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:58:07.467971 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d26e341-e2c3-41be-8d72-69cbf5944555 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:07.469414 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d26e341-e2c3-41be-8d72-69cbf5944555 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:07.482458 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 636/1245] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 6 16:58:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:58:07.488538 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea340b-73f7-42bc-b0a0-ac8907aa59a6 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:07.585619 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea340b-73f7-42bc-b0a0-ac8907aa59a6 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:07.620237 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4136609-90ab-4251-b845-d6f53deb6725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:07.623542 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4136609-90ab-4251-b845-d6f53deb6725 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:07.623542 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4136609-90ab-4251-b845-d6f53deb6725 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:07.623542 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4136609-90ab-4251-b845-d6f53deb6725 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:07.623542 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4136609-90ab-4251-b845-d6f53deb6725 None admin] RBAC: Adding query filter params (role.id=750bca9c32ac4aa49adcbee30463f1fa, scope.project.id=7938b0346d4240ab89774327d09a89ae, user.id=76ca877bee9041d28ad41e45ef3dd18c) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:58:07.623542 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4136609-90ab-4251-b845-d6f53deb6725 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:07.624114 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4136609-90ab-4251-b845-d6f53deb6725 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:07.629960 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 610/1246] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:58:07 2022] GET /identity/v3/role_assignments?user.id=76ca877bee9041d28ad41e45ef3dd18c&scope.project.id=7938b0346d4240ab89774327d09a89ae&role.id=750bca9c32ac4aa49adcbee30463f1fa => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:58:09.703609 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ed0d0-000c-479c-9942-d560cac7c7ac None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:09.704209 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ed0d0-000c-479c-9942-d560cac7c7ac None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:09.704209 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ed0d0-000c-479c-9942-d560cac7c7ac None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:09.705550 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 637/1247] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:58:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:58:09.711158 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a685e36-d8e8-425c-921e-21049e7ae041 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:09.711599 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a685e36-d8e8-425c-921e-21049e7ae041 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:09.711770 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a685e36-d8e8-425c-921e-21049e7ae041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:09.716757 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:09.728760 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-3a685e36-d8e8-425c-921e-21049e7ae041 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:58:09.730007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-3a685e36-d8e8-425c-921e-21049e7ae041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:58:09.798386 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 611/1248] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:58:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:58:09.822152 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779ed0d0-000c-479c-9942-d560cac7c7ac None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:09.840394 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:09.920795 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779ed0d0-000c-479c-9942-d560cac7c7ac None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:09.952725 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0183d25-9286-4857-aea8-3f261f14b381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:09.953596 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0183d25-9286-4857-aea8-3f261f14b381 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:09.953957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0183d25-9286-4857-aea8-3f261f14b381 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:09.954181 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0183d25-9286-4857-aea8-3f261f14b381 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:09.965674 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0183d25-9286-4857-aea8-3f261f14b381 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:58:09.973349 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-a0183d25-9286-4857-aea8-3f261f14b381 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:58:09.973349 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:58:09.973349 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:58:09.973349 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:58:09.973349 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:58:09.973349 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:58:09.973349 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:58:09.973349 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:58:09.973349 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:58:09.973349 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:58:09.973349 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:58:09.973349 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:58:09.973349 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:58:09.973349 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:58:09.973349 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:58:09.973349 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:58:09.974469 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:58:09.974469 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:58:09.974469 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:58:09.974469 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:58:09.974469 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:58:09.974469 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:58:09.974469 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:58:09.974469 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:58:09.974469 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:58:09.974469 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:58:09.974469 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:58:09.974469 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:58:09.974469 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:58:09.974469 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:58:09.974469 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:58:09.974469 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:58:09.975342 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:58:09.975342 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:58:09.975342 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:58:09.975342 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:58:09.975342 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:58:09.975342 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:58:09.975342 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:58:09.975342 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:58:09.975342 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:58:09.975342 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:58:09.975342 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:58:09.975342 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:58:09.975342 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 638/1249] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 6 16:58:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:58:09.980614 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a685e36-d8e8-425c-921e-21049e7ae041 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:10.076222 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a685e36-d8e8-425c-921e-21049e7ae041 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:10.104309 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:10.108858 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d8015-80e6-43a4-b202-29df4c49e67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:10.109538 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d8015-80e6-43a4-b202-29df4c49e67b None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:10.109824 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d8015-80e6-43a4-b202-29df4c49e67b None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:10.110103 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d8015-80e6-43a4-b202-29df4c49e67b None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:10.110844 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780d8015-80e6-43a4-b202-29df4c49e67b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:58:10.111153 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780d8015-80e6-43a4-b202-29df4c49e67b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:10.112546 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780d8015-80e6-43a4-b202-29df4c49e67b None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:10.120939 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 612/1250] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:58:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:58:10.126683 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0183d25-9286-4857-aea8-3f261f14b381 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:10.203197 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:10.225706 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0183d25-9286-4857-aea8-3f261f14b381 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:10.259271 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febdb692-8451-4667-acd2-ffaa1f3575ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:10.260091 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febdb692-8451-4667-acd2-ffaa1f3575ac None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:10.260803 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febdb692-8451-4667-acd2-ffaa1f3575ac None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:10.261079 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febdb692-8451-4667-acd2-ffaa1f3575ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:10.271749 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febdb692-8451-4667-acd2-ffaa1f3575ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:10.273187 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febdb692-8451-4667-acd2-ffaa1f3575ac None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:10.289928 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-febdb692-8451-4667-acd2-ffaa1f3575ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:58:10.289928 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:58:10.289928 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:58:10.289928 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:58:10.289928 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:58:10.289928 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:58:10.289928 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:58:10.289928 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:58:10.289928 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:58:10.289928 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:58:10.289928 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:58:10.289928 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:58:10.289928 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:58:10.289928 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:58:10.289928 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:58:10.291922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:58:10.291922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:58:10.291922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:58:10.291922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:58:10.291922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:58:10.291922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:58:10.291922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:58:10.291922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:58:10.291922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:58:10.291922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:58:10.291922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:58:10.291922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:58:10.291922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:58:10.291922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:58:10.291922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:58:10.291922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:58:10.291922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:58:10.292850 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:58:10.292850 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:58:10.292850 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:58:10.292850 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:58:10.292850 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 639/1251] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:58:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:58:10.296854 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d8015-80e6-43a4-b202-29df4c49e67b None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:10.395042 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d8015-80e6-43a4-b202-29df4c49e67b None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:10.427186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ca7d0-5b82-494d-be12-758e55fd9c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:10.428358 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ca7d0-5b82-494d-be12-758e55fd9c23 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:10.428640 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ca7d0-5b82-494d-be12-758e55fd9c23 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:10.428911 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ca7d0-5b82-494d-be12-758e55fd9c23 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:10.429623 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158ca7d0-5b82-494d-be12-758e55fd9c23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:58:10.429925 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158ca7d0-5b82-494d-be12-758e55fd9c23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:10.431281 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158ca7d0-5b82-494d-be12-758e55fd9c23 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:10.442854 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 613/1252] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:58:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:58:10.448960 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febdb692-8451-4667-acd2-ffaa1f3575ac None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:10.547234 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febdb692-8451-4667-acd2-ffaa1f3575ac None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:10.572764 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:10.580261 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ca59f-1a2c-4523-9ac5-dbb89e5da16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:10.581051 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ca59f-1a2c-4523-9ac5-dbb89e5da16a None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:10.581339 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ca59f-1a2c-4523-9ac5-dbb89e5da16a None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:10.581619 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63ca59f-1a2c-4523-9ac5-dbb89e5da16a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:10.588834 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ca59f-1a2c-4523-9ac5-dbb89e5da16a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:10.590242 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63ca59f-1a2c-4523-9ac5-dbb89e5da16a None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:10.604643 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-f63ca59f-1a2c-4523-9ac5-dbb89e5da16a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 06 16:58:10.604643 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:58:10.604643 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:58:10.604643 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:58:10.604643 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:58:10.604643 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:58:10.604643 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:58:10.604643 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:58:10.604643 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:58:10.604643 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:58:10.604643 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:58:10.604643 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:58:10.604643 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 16:58:10.604643 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 16:58:10.604643 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 16:58:10.604643 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 16:58:10.605809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:58:10.605809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:58:10.605809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 16:58:10.605809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:58:10.605809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 16:58:10.605809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:58:10.605809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:58:10.605809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:58:10.605809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:58:10.605809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:58:10.605809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:58:10.605809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:58:10.605809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:58:10.605809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:58:10.605809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:58:10.605809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:58:10.605809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:58:10.605809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:58:10.606973 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:58:10.606973 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:58:10.606973 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 16:58:10.606973 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 16:58:10.606973 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 16:58:10.606973 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 16:58:10.606973 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 16:58:10.606973 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 16:58:10.606973 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 06 16:58:10.606973 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:58:10.606973 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 640/1253] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:58:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:58:10.611609 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ca7d0-5b82-494d-be12-758e55fd9c23 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:10.666280 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:10.710312 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ca7d0-5b82-494d-be12-758e55fd9c23 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:10.743791 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e11bca3-f06b-4606-96ce-7cac7fe950ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:10.744468 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11bca3-f06b-4606-96ce-7cac7fe950ab None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:10.744750 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11bca3-f06b-4606-96ce-7cac7fe950ab None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:10.745023 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e11bca3-f06b-4606-96ce-7cac7fe950ab None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:10.745853 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e11bca3-f06b-4606-96ce-7cac7fe950ab None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:58:10.746157 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e11bca3-f06b-4606-96ce-7cac7fe950ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:10.747638 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e11bca3-f06b-4606-96ce-7cac7fe950ab None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:10.764213 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 614/1254] 10.222.0.40 () {56 vars in 1143 bytes} [Wed Jul 6 16:58:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:58:10.769982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ca59f-1a2c-4523-9ac5-dbb89e5da16a None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:10.868868 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63ca59f-1a2c-4523-9ac5-dbb89e5da16a None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:10.902133 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15061e-b56f-491d-b59b-e4b73204136a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:10.902787 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15061e-b56f-491d-b59b-e4b73204136a None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:10.903183 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15061e-b56f-491d-b59b-e4b73204136a None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:10.903462 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf15061e-b56f-491d-b59b-e4b73204136a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:10.957003 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:10.969212 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf15061e-b56f-491d-b59b-e4b73204136a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:10.970659 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf15061e-b56f-491d-b59b-e4b73204136a None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:11.066953 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 641/1255] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:58:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 16:58:11.072860 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e11bca3-f06b-4606-96ce-7cac7fe950ab None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:11.170514 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e11bca3-f06b-4606-96ce-7cac7fe950ab None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:11.183982 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:11.204622 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6142c785-af5f-4dc0-a680-8a4cd1d980a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:11.205611 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142c785-af5f-4dc0-a680-8a4cd1d980a1 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:11.205892 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142c785-af5f-4dc0-a680-8a4cd1d980a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:11.206165 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142c785-af5f-4dc0-a680-8a4cd1d980a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:11.216473 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6142c785-af5f-4dc0-a680-8a4cd1d980a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:11.217901 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6142c785-af5f-4dc0-a680-8a4cd1d980a1 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:11.235438 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-6142c785-af5f-4dc0-a680-8a4cd1d980a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:58:11.235438 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:58:11.235438 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:58:11.235438 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:58:11.235438 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:58:11.235438 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:58:11.235438 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:58:11.235438 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:58:11.235438 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:58:11.235438 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:58:11.235438 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:58:11.235438 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:58:11.235438 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:58:11.235438 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:58:11.235438 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:58:11.243186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:58:11.243186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:58:11.243186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:58:11.243186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:58:11.243186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:58:11.243186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:58:11.243186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:58:11.243186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:58:11.243186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:58:11.243186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:58:11.243186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:58:11.243186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:58:11.243186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:58:11.243186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:58:11.243186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:58:11.243186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:58:11.243186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:58:11.245287 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:58:11.245287 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:58:11.245287 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:58:11.245287 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:58:11.245287 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 615/1256] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:58:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:58:11.245910 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15061e-b56f-491d-b59b-e4b73204136a None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:11.330382 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:11.342380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf15061e-b56f-491d-b59b-e4b73204136a None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:11.374411 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87a4396-cee6-4ebd-a076-9b6368bca91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:11.375076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a4396-cee6-4ebd-a076-9b6368bca91e None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:11.375360 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a4396-cee6-4ebd-a076-9b6368bca91e None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:11.375636 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a4396-cee6-4ebd-a076-9b6368bca91e None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:11.376386 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87a4396-cee6-4ebd-a076-9b6368bca91e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:58:11.376693 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87a4396-cee6-4ebd-a076-9b6368bca91e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:11.378062 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87a4396-cee6-4ebd-a076-9b6368bca91e None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:11.389439 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 642/1257] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:58:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:58:11.395157 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6142c785-af5f-4dc0-a680-8a4cd1d980a1 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:11.492311 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6142c785-af5f-4dc0-a680-8a4cd1d980a1 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:11.526523 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d199f9a4-d846-4ae7-809c-b1d119dc33bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:11.527316 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d199f9a4-d846-4ae7-809c-b1d119dc33bb None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:11.527598 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d199f9a4-d846-4ae7-809c-b1d119dc33bb None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:11.527904 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d199f9a4-d846-4ae7-809c-b1d119dc33bb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:11.534647 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d199f9a4-d846-4ae7-809c-b1d119dc33bb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:11.536110 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d199f9a4-d846-4ae7-809c-b1d119dc33bb None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:11.549777 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-d199f9a4-d846-4ae7-809c-b1d119dc33bb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:58:11.549777 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:58:11.549777 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:58:11.549777 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:58:11.549777 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:58:11.549777 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:58:11.549777 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:58:11.549777 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:58:11.549777 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:58:11.549777 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:58:11.549777 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:58:11.549777 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:58:11.549777 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 16:58:11.549777 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 16:58:11.549777 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 16:58:11.550917 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 16:58:11.550917 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:58:11.550917 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:58:11.550917 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:58:11.550917 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:58:11.550917 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:58:11.550917 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:58:11.550917 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:58:11.550917 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:58:11.550917 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:58:11.550917 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:58:11.550917 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:58:11.550917 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:58:11.550917 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:58:11.550917 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:58:11.550917 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:58:11.550917 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:58:11.551975 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 16:58:11.551975 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 16:58:11.551975 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 16:58:11.551975 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 16:58:11.551975 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 16:58:11.551975 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 16:58:11.551975 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:58:11.551975 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:58:11.551975 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 616/1258] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:58:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:58:11.557471 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87a4396-cee6-4ebd-a076-9b6368bca91e None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:11.656934 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87a4396-cee6-4ebd-a076-9b6368bca91e None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:11.689772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f02b02d-37dc-4c49-8f34-92fe1e6fc1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:11.690433 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02b02d-37dc-4c49-8f34-92fe1e6fc1fa None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:11.690723 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02b02d-37dc-4c49-8f34-92fe1e6fc1fa None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:11.690999 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f02b02d-37dc-4c49-8f34-92fe1e6fc1fa None admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:11.691805 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f02b02d-37dc-4c49-8f34-92fe1e6fc1fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:58:11.692117 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f02b02d-37dc-4c49-8f34-92fe1e6fc1fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:11.694152 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f02b02d-37dc-4c49-8f34-92fe1e6fc1fa None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:11.705883 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 643/1259] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 6 16:58:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:58:11.711992 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d199f9a4-d846-4ae7-809c-b1d119dc33bb None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:11.737122 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:11.810094 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d199f9a4-d846-4ae7-809c-b1d119dc33bb None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:11.843774 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd58f44-ec39-49b5-8e98-5494fd9326cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:11.844432 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd58f44-ec39-49b5-8e98-5494fd9326cb None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:11.844720 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd58f44-ec39-49b5-8e98-5494fd9326cb None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:11.845000 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd58f44-ec39-49b5-8e98-5494fd9326cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:11.845832 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd58f44-ec39-49b5-8e98-5494fd9326cb None admin] RBAC: Adding query filter params (role.id=b41f8a7a58e64f749f2099ca45cc04a7, scope.project.id=7938b0346d4240ab89774327d09a89ae, user.id=76ca877bee9041d28ad41e45ef3dd18c) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:58:11.846143 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd58f44-ec39-49b5-8e98-5494fd9326cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:11.847594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd58f44-ec39-49b5-8e98-5494fd9326cb None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:11.852954 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 617/1260] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:58:11 2022] GET /identity/v3/role_assignments?user.id=76ca877bee9041d28ad41e45ef3dd18c&scope.project.id=7938b0346d4240ab89774327d09a89ae&role.id=b41f8a7a58e64f749f2099ca45cc04a7 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:58:13.860116 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b210a2b-4ceb-4aba-8098-60db3d70a23a None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:13.860667 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b210a2b-4ceb-4aba-8098-60db3d70a23a None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:13.860667 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b210a2b-4ceb-4aba-8098-60db3d70a23a None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:13.862038 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 644/1261] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:58:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:58:13.867168 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661b1a6-78db-4fdc-9981-ce1b9e2c4903 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:13.867604 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661b1a6-78db-4fdc-9981-ce1b9e2c4903 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:13.867747 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661b1a6-78db-4fdc-9981-ce1b9e2c4903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:13.884269 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-c661b1a6-78db-4fdc-9981-ce1b9e2c4903 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:58:13.885443 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c661b1a6-78db-4fdc-9981-ce1b9e2c4903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:58:13.952599 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 618/1262] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:58:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:58:13.976466 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b210a2b-4ceb-4aba-8098-60db3d70a23a None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:13.987293 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:14.075838 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b210a2b-4ceb-4aba-8098-60db3d70a23a None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:14.110166 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22056ea3-bc38-49ba-9a67-7c9e76fee1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:14.110962 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22056ea3-bc38-49ba-9a67-7c9e76fee1b8 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:14.111249 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22056ea3-bc38-49ba-9a67-7c9e76fee1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:14.111527 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22056ea3-bc38-49ba-9a67-7c9e76fee1b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:14.121781 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22056ea3-bc38-49ba-9a67-7c9e76fee1b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:58:14.129499 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-22056ea3-bc38-49ba-9a67-7c9e76fee1b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:58:14.129499 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:58:14.129499 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:58:14.129499 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:58:14.129499 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:58:14.129499 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:58:14.129499 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:58:14.129499 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:58:14.129499 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:58:14.129499 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:58:14.129499 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:58:14.129499 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:58:14.129499 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:58:14.129499 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:58:14.129499 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:58:14.129499 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:58:14.130633 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:58:14.130633 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:58:14.130633 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:58:14.130633 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:58:14.130633 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:58:14.130633 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:58:14.130633 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:58:14.130633 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:58:14.130633 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:58:14.130633 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:58:14.130633 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:58:14.130633 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:58:14.130633 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:58:14.130633 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:58:14.130633 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:58:14.130633 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:58:14.131680 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:58:14.131680 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:58:14.131680 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:58:14.131680 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:58:14.131680 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:58:14.131680 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:58:14.131680 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:58:14.131680 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:58:14.131680 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:58:14.131680 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:58:14.131680 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:58:14.131680 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:58:14.131680 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 645/1263] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 6 16:58:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:58:14.138818 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c661b1a6-78db-4fdc-9981-ce1b9e2c4903 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:14.157518 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:14.234140 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c661b1a6-78db-4fdc-9981-ce1b9e2c4903 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:14.266078 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b3a3b-d89a-4234-bc5f-68b00a817503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:14.266719 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b3a3b-d89a-4234-bc5f-68b00a817503 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:14.267017 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b3a3b-d89a-4234-bc5f-68b00a817503 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:14.267301 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b3a3b-d89a-4234-bc5f-68b00a817503 None admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:14.268127 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b3a3b-d89a-4234-bc5f-68b00a817503 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:58:14.280024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b3a3b-d89a-4234-bc5f-68b00a817503 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:14.280024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7b3a3b-d89a-4234-bc5f-68b00a817503 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:14.280024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 619/1264] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:58:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:58:14.284189 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22056ea3-bc38-49ba-9a67-7c9e76fee1b8 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:14.350102 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:14.382860 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22056ea3-bc38-49ba-9a67-7c9e76fee1b8 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:14.415365 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d48faf8-4960-4031-aac7-541d600a96fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:14.416258 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d48faf8-4960-4031-aac7-541d600a96fd None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:14.416540 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d48faf8-4960-4031-aac7-541d600a96fd None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:14.416814 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d48faf8-4960-4031-aac7-541d600a96fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:14.426934 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d48faf8-4960-4031-aac7-541d600a96fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:14.430658 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d48faf8-4960-4031-aac7-541d600a96fd None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:14.445613 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-8d48faf8-4960-4031-aac7-541d600a96fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:58:14.445613 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:58:14.445613 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:58:14.445613 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:58:14.445613 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:58:14.445613 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:58:14.445613 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:58:14.445613 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:58:14.445613 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:58:14.445613 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:58:14.445613 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:58:14.445613 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:58:14.445613 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:58:14.445613 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:58:14.445613 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:58:14.452524 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:58:14.452524 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:58:14.452524 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:58:14.452524 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:58:14.452524 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:58:14.452524 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:58:14.452524 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:58:14.452524 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:58:14.452524 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:58:14.452524 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:58:14.452524 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:58:14.452524 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:58:14.452524 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:58:14.452524 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:58:14.452524 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:58:14.452524 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:58:14.452524 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:58:14.453718 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:58:14.453718 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:58:14.453718 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:58:14.453718 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:58:14.453718 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 646/1265] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:58:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:58:14.454260 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b3a3b-d89a-4234-bc5f-68b00a817503 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:14.537398 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:14.555485 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b3a3b-d89a-4234-bc5f-68b00a817503 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:14.587474 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf5444-627d-489b-98e9-9ab80dc5ce7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:14.588144 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf5444-627d-489b-98e9-9ab80dc5ce7c None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:14.588425 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf5444-627d-489b-98e9-9ab80dc5ce7c None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:14.588699 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf5444-627d-489b-98e9-9ab80dc5ce7c None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:14.589405 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cf5444-627d-489b-98e9-9ab80dc5ce7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:58:14.589707 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cf5444-627d-489b-98e9-9ab80dc5ce7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:14.591073 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cf5444-627d-489b-98e9-9ab80dc5ce7c None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:14.601746 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 620/1266] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:58:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:58:14.607645 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d48faf8-4960-4031-aac7-541d600a96fd None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:14.706947 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d48faf8-4960-4031-aac7-541d600a96fd None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:14.725657 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:14.738918 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41201394-d9fe-4a23-a9ff-40512d960c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:14.739716 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41201394-d9fe-4a23-a9ff-40512d960c36 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:14.740556 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41201394-d9fe-4a23-a9ff-40512d960c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:14.740832 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41201394-d9fe-4a23-a9ff-40512d960c36 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:14.748069 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41201394-d9fe-4a23-a9ff-40512d960c36 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:14.749500 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41201394-d9fe-4a23-a9ff-40512d960c36 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:14.764011 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-41201394-d9fe-4a23-a9ff-40512d960c36 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 06 16:58:14.764011 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:58:14.764011 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:58:14.764011 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:58:14.764011 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:58:14.764011 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:58:14.764011 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:58:14.764011 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:58:14.764011 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:58:14.764011 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:58:14.764011 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:58:14.764011 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:58:14.764011 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 16:58:14.764011 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 16:58:14.764011 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 16:58:14.764011 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 16:58:14.765146 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:58:14.765146 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:58:14.765146 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 16:58:14.765146 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:58:14.765146 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 16:58:14.765146 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:58:14.765146 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:58:14.765146 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:58:14.765146 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:58:14.765146 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:58:14.765146 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:58:14.765146 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:58:14.765146 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:58:14.765146 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:58:14.765146 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:58:14.765146 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:58:14.765146 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:58:14.765146 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:58:14.766162 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:58:14.766162 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:58:14.766162 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 16:58:14.766162 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 16:58:14.766162 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 16:58:14.766162 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 16:58:14.766162 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 16:58:14.766162 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 16:58:14.766162 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 06 16:58:14.766162 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:58:14.766162 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 647/1267] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:58:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:58:14.771328 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf5444-627d-489b-98e9-9ab80dc5ce7c None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:14.873695 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf5444-627d-489b-98e9-9ab80dc5ce7c None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:14.906066 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d435532-358d-4512-9d31-5a769563dc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:14.906755 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d435532-358d-4512-9d31-5a769563dc4e None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:14.907037 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d435532-358d-4512-9d31-5a769563dc4e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:14.907310 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d435532-358d-4512-9d31-5a769563dc4e None admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:14.908162 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d435532-358d-4512-9d31-5a769563dc4e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:58:14.908470 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d435532-358d-4512-9d31-5a769563dc4e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:14.909887 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d435532-358d-4512-9d31-5a769563dc4e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:14.918626 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:14.926264 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 621/1268] 10.222.0.40 () {56 vars in 1143 bytes} [Wed Jul 6 16:58:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:58:14.932540 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41201394-d9fe-4a23-a9ff-40512d960c36 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:15.031473 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41201394-d9fe-4a23-a9ff-40512d960c36 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:15.065711 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e2270a-6bc9-4be6-a6f5-bb29771d7ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:15.066603 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2270a-6bc9-4be6-a6f5-bb29771d7ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:15.066888 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2270a-6bc9-4be6-a6f5-bb29771d7ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:15.067157 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e2270a-6bc9-4be6-a6f5-bb29771d7ce4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:15.077643 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e2270a-6bc9-4be6-a6f5-bb29771d7ce4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:15.079095 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e2270a-6bc9-4be6-a6f5-bb29771d7ce4 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:15.095655 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-68e2270a-6bc9-4be6-a6f5-bb29771d7ce4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:58:15.095655 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:58:15.095655 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:58:15.095655 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:58:15.095655 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:58:15.095655 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:58:15.095655 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:58:15.095655 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:58:15.095655 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:58:15.095655 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:58:15.095655 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:58:15.095655 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:58:15.095655 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:58:15.095655 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:58:15.095655 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:58:15.097046 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:58:15.097046 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:58:15.097046 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:58:15.097046 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:58:15.097046 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:58:15.097046 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:58:15.097046 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:58:15.097046 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:58:15.097046 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:58:15.097046 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:58:15.097046 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:58:15.097046 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:58:15.097046 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:58:15.097046 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:58:15.097046 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:58:15.097046 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:58:15.097046 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:58:15.097951 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:58:15.097951 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:58:15.097951 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:58:15.097951 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:58:15.097951 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 648/1269] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:58:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:58:15.103103 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d435532-358d-4512-9d31-5a769563dc4e None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:15.198828 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d435532-358d-4512-9d31-5a769563dc4e None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:15.231631 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a05c4-6a21-4b76-8f5b-19bcab0e79a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:15.232473 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a05c4-6a21-4b76-8f5b-19bcab0e79a9 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:15.232764 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a05c4-6a21-4b76-8f5b-19bcab0e79a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:15.233039 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5a05c4-6a21-4b76-8f5b-19bcab0e79a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:15.233761 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5a05c4-6a21-4b76-8f5b-19bcab0e79a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:58:15.234067 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5a05c4-6a21-4b76-8f5b-19bcab0e79a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:15.235449 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5a05c4-6a21-4b76-8f5b-19bcab0e79a9 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:15.247298 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 622/1270] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:58:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:58:15.252936 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e2270a-6bc9-4be6-a6f5-bb29771d7ce4 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:15.283628 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:15.350248 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e2270a-6bc9-4be6-a6f5-bb29771d7ce4 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:15.382747 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f5084-81df-4b45-b988-57fb38e26c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:15.383537 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f5084-81df-4b45-b988-57fb38e26c88 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:15.383851 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f5084-81df-4b45-b988-57fb38e26c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:15.384131 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661f5084-81df-4b45-b988-57fb38e26c88 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:15.390695 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661f5084-81df-4b45-b988-57fb38e26c88 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:15.392162 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661f5084-81df-4b45-b988-57fb38e26c88 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:15.406408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-661f5084-81df-4b45-b988-57fb38e26c88 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:58:15.406408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:58:15.406408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:58:15.406408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:58:15.406408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:58:15.406408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:58:15.406408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:58:15.406408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:58:15.406408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:58:15.406408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:58:15.406408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:58:15.406408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:58:15.406408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 16:58:15.406408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 16:58:15.406408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 16:58:15.407434 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 16:58:15.407434 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:58:15.407434 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:58:15.407434 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:58:15.407434 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:58:15.407434 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:58:15.407434 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:58:15.407434 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:58:15.407434 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:58:15.407434 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:58:15.407434 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:58:15.407434 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:58:15.407434 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:58:15.407434 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:58:15.407434 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:58:15.407434 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:58:15.407434 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:58:15.408571 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 16:58:15.408571 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 16:58:15.408571 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 16:58:15.408571 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 16:58:15.408571 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 16:58:15.408571 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 16:58:15.408571 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:58:15.408571 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:58:15.408571 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 649/1271] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:58:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:58:15.413969 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a05c4-6a21-4b76-8f5b-19bcab0e79a9 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:15.470627 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:15.510607 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5a05c4-6a21-4b76-8f5b-19bcab0e79a9 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:15.544751 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dfe1ac-1aa1-4984-8b1a-506007b7df1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:15.545413 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dfe1ac-1aa1-4984-8b1a-506007b7df1f None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:15.545701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dfe1ac-1aa1-4984-8b1a-506007b7df1f None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:15.545979 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dfe1ac-1aa1-4984-8b1a-506007b7df1f None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:15.546765 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dfe1ac-1aa1-4984-8b1a-506007b7df1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:58:15.547076 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dfe1ac-1aa1-4984-8b1a-506007b7df1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:15.548600 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dfe1ac-1aa1-4984-8b1a-506007b7df1f None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:15.561231 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 623/1272] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 6 16:58:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:58:15.567186 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f5084-81df-4b45-b988-57fb38e26c88 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:15.657527 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:15.666472 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661f5084-81df-4b45-b988-57fb38e26c88 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:15.705638 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81aa527-618e-48cc-bf30-284108e032dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:15.706528 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81aa527-618e-48cc-bf30-284108e032dc None admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:15.706801 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81aa527-618e-48cc-bf30-284108e032dc None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:15.707070 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81aa527-618e-48cc-bf30-284108e032dc None admin] PATH_INFO: `/v3/projects/7938b0346d4240ab89774327d09a89ae/users/76ca877bee9041d28ad41e45ef3dd18c/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:15.709588 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81aa527-618e-48cc-bf30-284108e032dc None admin] RBAC: Authorizing `identity:create_grant(project_id=7938b0346d4240ab89774327d09a89ae, user_id=76ca877bee9041d28ad41e45ef3dd18c, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:15.711007 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81aa527-618e-48cc-bf30-284108e032dc None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:15.836314 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 650/1273] 10.222.0.40 () {58 vars in 1439 bytes} [Wed Jul 6 16:58:15 2022] PUT /identity/v3/projects/7938b0346d4240ab89774327d09a89ae/users/76ca877bee9041d28ad41e45ef3dd18c/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 16:58:17.836009 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c4dd1-7545-46b5-93b7-f09632ddf14d None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:17.836564 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c4dd1-7545-46b5-93b7-f09632ddf14d None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:17.836564 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06c4dd1-7545-46b5-93b7-f09632ddf14d None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:17.839760 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 624/1274] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:58:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:58:17.843166 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9de177-cb3b-4c7e-8db4-61b0f961c3fd None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:17.843565 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9de177-cb3b-4c7e-8db4-61b0f961c3fd None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:17.843721 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9de177-cb3b-4c7e-8db4-61b0f961c3fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:17.860615 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-eb9de177-cb3b-4c7e-8db4-61b0f961c3fd None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:58:17.861776 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-eb9de177-cb3b-4c7e-8db4-61b0f961c3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:58:17.927979 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 651/1275] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:58:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:58:17.951440 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c4dd1-7545-46b5-93b7-f09632ddf14d None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:18.056224 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06c4dd1-7545-46b5-93b7-f09632ddf14d None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:18.073465 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:18.090057 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bbe063-a331-4d60-b60e-ab887935da60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:18.090844 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bbe063-a331-4d60-b60e-ab887935da60 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:18.091124 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bbe063-a331-4d60-b60e-ab887935da60 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:18.091400 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bbe063-a331-4d60-b60e-ab887935da60 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:18.101104 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bbe063-a331-4d60-b60e-ab887935da60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 06 16:58:18.108610 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-f8bbe063-a331-4d60-b60e-ab887935da60 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:58:18.108610 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:58:18.108610 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:58:18.108610 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:58:18.108610 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:58:18.108610 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:58:18.108610 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:58:18.108610 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:58:18.108610 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:58:18.108610 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:58:18.108610 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:58:18.108610 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:58:18.108610 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 06 16:58:18.108610 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 06 16:58:18.108610 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 06 16:58:18.108610 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 06 16:58:18.109722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 06 16:58:18.109722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 06 16:58:18.109722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 06 16:58:18.109722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 06 16:58:18.109722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:58:18.109722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:58:18.109722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:58:18.109722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:58:18.109722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:58:18.109722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:58:18.109722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:58:18.109722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:58:18.109722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:58:18.109722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:58:18.109722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:58:18.109722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:58:18.110587 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:58:18.110587 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:58:18.110587 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:58:18.110587 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:58:18.110587 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 06 16:58:18.110587 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 06 16:58:18.110587 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 06 16:58:18.110587 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 06 16:58:18.110587 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 06 16:58:18.110587 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 06 16:58:18.110587 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 06 16:58:18.110587 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:58:18.110587 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 625/1276] 10.222.0.40 () {56 vars in 1096 bytes} [Wed Jul 6 16:58:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:58:18.115930 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9de177-cb3b-4c7e-8db4-61b0f961c3fd None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:18.197819 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:18.215931 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9de177-cb3b-4c7e-8db4-61b0f961c3fd None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:18.248234 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c61c70-b39f-4d37-8559-0b44b4f18e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:18.248903 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c61c70-b39f-4d37-8559-0b44b4f18e15 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:18.249191 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c61c70-b39f-4d37-8559-0b44b4f18e15 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:18.249470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c61c70-b39f-4d37-8559-0b44b4f18e15 None admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:18.250236 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c61c70-b39f-4d37-8559-0b44b4f18e15 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:58:18.250523 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c61c70-b39f-4d37-8559-0b44b4f18e15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:18.251955 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c61c70-b39f-4d37-8559-0b44b4f18e15 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:18.260359 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 652/1277] 10.222.0.40 () {56 vars in 1099 bytes} [Wed Jul 6 16:58:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:58:18.266066 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bbe063-a331-4d60-b60e-ab887935da60 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:18.363453 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bbe063-a331-4d60-b60e-ab887935da60 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:18.399469 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e229c7dc-e0de-4fa5-976d-dabc4668ff8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:18.400346 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e229c7dc-e0de-4fa5-976d-dabc4668ff8e None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:18.400631 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e229c7dc-e0de-4fa5-976d-dabc4668ff8e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:18.400905 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e229c7dc-e0de-4fa5-976d-dabc4668ff8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:18.411431 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e229c7dc-e0de-4fa5-976d-dabc4668ff8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:18.413007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e229c7dc-e0de-4fa5-976d-dabc4668ff8e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:18.430539 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-e229c7dc-e0de-4fa5-976d-dabc4668ff8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:58:18.430539 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:58:18.430539 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:58:18.430539 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:58:18.430539 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:58:18.430539 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:58:18.430539 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:58:18.430539 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:58:18.430539 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:58:18.430539 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:58:18.430539 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:58:18.430539 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:58:18.430539 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:58:18.430539 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:58:18.430539 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:58:18.431593 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:58:18.431593 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:58:18.431593 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:58:18.431593 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:58:18.431593 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:58:18.431593 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:58:18.431593 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:58:18.431593 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:58:18.431593 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:58:18.431593 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:58:18.431593 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:58:18.431593 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:58:18.431593 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:58:18.431593 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:58:18.431593 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:58:18.431593 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:58:18.431593 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:58:18.432727 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:58:18.432727 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:58:18.432727 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:58:18.432727 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:58:18.432727 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 626/1278] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:58:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:58:18.437940 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c61c70-b39f-4d37-8559-0b44b4f18e15 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:18.535954 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c61c70-b39f-4d37-8559-0b44b4f18e15 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:18.568191 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6326824-33fd-4597-99b1-91d79ad442ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:18.568875 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6326824-33fd-4597-99b1-91d79ad442ca None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:18.569155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6326824-33fd-4597-99b1-91d79ad442ca None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:18.569429 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6326824-33fd-4597-99b1-91d79ad442ca None admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:18.570141 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6326824-33fd-4597-99b1-91d79ad442ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:58:18.570446 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6326824-33fd-4597-99b1-91d79ad442ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:18.571820 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6326824-33fd-4597-99b1-91d79ad442ca None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:18.583274 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 653/1279] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:58:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:58:18.589642 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e229c7dc-e0de-4fa5-976d-dabc4668ff8e None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:18.620289 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:18.691958 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e229c7dc-e0de-4fa5-976d-dabc4668ff8e None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:18.725834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c453fe3-3b98-4670-b54e-01f629560d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:18.726632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c453fe3-3b98-4670-b54e-01f629560d7d None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:18.726913 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c453fe3-3b98-4670-b54e-01f629560d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:18.727187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c453fe3-3b98-4670-b54e-01f629560d7d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:18.734979 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c453fe3-3b98-4670-b54e-01f629560d7d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:18.736565 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c453fe3-3b98-4670-b54e-01f629560d7d None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:18.751298 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-0c453fe3-3b98-4670-b54e-01f629560d7d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 06 16:58:18.751298 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:58:18.751298 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:58:18.751298 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:58:18.751298 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:58:18.751298 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:58:18.751298 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:58:18.751298 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:58:18.751298 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:58:18.751298 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:58:18.751298 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:58:18.751298 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:58:18.751298 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 06 16:58:18.751298 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 06 16:58:18.751298 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 06 16:58:18.751298 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 06 16:58:18.752942 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:58:18.752942 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:58:18.752942 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 06 16:58:18.752942 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:58:18.752942 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 06 16:58:18.752942 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 06 16:58:18.752942 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:58:18.752942 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:58:18.752942 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:58:18.752942 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:58:18.752942 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:58:18.752942 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:58:18.752942 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:58:18.752942 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:58:18.752942 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:58:18.752942 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:58:18.752942 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:58:18.752942 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:58:18.754388 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:58:18.754388 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:58:18.754388 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 06 16:58:18.754388 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 06 16:58:18.754388 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 06 16:58:18.754388 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 06 16:58:18.754388 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 06 16:58:18.754388 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 06 16:58:18.754388 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 06 16:58:18.754388 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:58:18.754388 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 627/1280] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:58:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:58:18.758964 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6326824-33fd-4597-99b1-91d79ad442ca None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:18.762670 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:18.859479 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6326824-33fd-4597-99b1-91d79ad442ca None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:18.892812 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a816042c-2b1d-4e72-a0a4-29aa74d4ba29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:18.893441 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a816042c-2b1d-4e72-a0a4-29aa74d4ba29 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:18.893728 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a816042c-2b1d-4e72-a0a4-29aa74d4ba29 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:18.894006 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a816042c-2b1d-4e72-a0a4-29aa74d4ba29 None admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:18.894844 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a816042c-2b1d-4e72-a0a4-29aa74d4ba29 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:58:18.895176 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a816042c-2b1d-4e72-a0a4-29aa74d4ba29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:18.897536 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a816042c-2b1d-4e72-a0a4-29aa74d4ba29 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:18.914655 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 654/1281] 10.222.0.40 () {56 vars in 1143 bytes} [Wed Jul 6 16:58:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:58:18.920359 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c453fe3-3b98-4670-b54e-01f629560d7d None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:19.008794 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:19.018236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c453fe3-3b98-4670-b54e-01f629560d7d None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:19.053004 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec491c16-322c-4155-a25d-943f04e0c472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:19.053653 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec491c16-322c-4155-a25d-943f04e0c472 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:19.053939 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec491c16-322c-4155-a25d-943f04e0c472 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:19.054214 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec491c16-322c-4155-a25d-943f04e0c472 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:19.119362 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec491c16-322c-4155-a25d-943f04e0c472 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:19.120839 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec491c16-322c-4155-a25d-943f04e0c472 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:19.206031 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:19.215661 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 628/1282] 10.222.0.40 () {58 vars in 1282 bytes} [Wed Jul 6 16:58:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 06 16:58:19.221455 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a816042c-2b1d-4e72-a0a4-29aa74d4ba29 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:19.281912 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:19.321220 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a816042c-2b1d-4e72-a0a4-29aa74d4ba29 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:19.355006 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62090c90-67d5-4471-9e1b-028eda81aa6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:19.355850 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62090c90-67d5-4471-9e1b-028eda81aa6e None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:19.356133 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62090c90-67d5-4471-9e1b-028eda81aa6e None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:19.356406 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62090c90-67d5-4471-9e1b-028eda81aa6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:19.367972 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62090c90-67d5-4471-9e1b-028eda81aa6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:19.369408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62090c90-67d5-4471-9e1b-028eda81aa6e None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:19.386418 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-62090c90-67d5-4471-9e1b-028eda81aa6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:58:19.386418 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:58:19.386418 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:58:19.386418 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:58:19.386418 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:58:19.386418 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:58:19.386418 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:58:19.386418 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:58:19.386418 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:58:19.386418 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:58:19.386418 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:58:19.386418 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:58:19.386418 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 06 16:58:19.386418 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 06 16:58:19.386418 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 06 16:58:19.387447 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 06 16:58:19.387447 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:58:19.387447 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:58:19.387447 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:58:19.387447 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:58:19.387447 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:58:19.387447 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:58:19.387447 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:58:19.387447 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:58:19.387447 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:58:19.387447 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:58:19.387447 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:58:19.387447 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:58:19.387447 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:58:19.387447 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:58:19.387447 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:58:19.387447 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:58:19.388684 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 06 16:58:19.388684 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 06 16:58:19.388684 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 06 16:58:19.388684 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:58:19.388684 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 655/1283] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 16:58:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:58:19.393314 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec491c16-322c-4155-a25d-943f04e0c472 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:19.490569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec491c16-322c-4155-a25d-943f04e0c472 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:19.523672 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff58353-b323-4c62-9701-be5a01e35418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:19.524363 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff58353-b323-4c62-9701-be5a01e35418 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:19.524651 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff58353-b323-4c62-9701-be5a01e35418 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:19.524931 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff58353-b323-4c62-9701-be5a01e35418 None admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:19.525651 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff58353-b323-4c62-9701-be5a01e35418 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:58:19.525957 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff58353-b323-4c62-9701-be5a01e35418 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:19.527323 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff58353-b323-4c62-9701-be5a01e35418 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:19.538783 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 629/1284] 10.222.0.40 () {56 vars in 1109 bytes} [Wed Jul 6 16:58:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:58:19.544422 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62090c90-67d5-4471-9e1b-028eda81aa6e None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:19.642229 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62090c90-67d5-4471-9e1b-028eda81aa6e None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:19.655650 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:19.675127 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76397da-e052-4760-b06d-104bf0b11e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:19.675948 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76397da-e052-4760-b06d-104bf0b11e91 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:19.676233 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76397da-e052-4760-b06d-104bf0b11e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:19.676508 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76397da-e052-4760-b06d-104bf0b11e91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:19.683458 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76397da-e052-4760-b06d-104bf0b11e91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:19.684943 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76397da-e052-4760-b06d-104bf0b11e91 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:19.698656 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application [None req-b76397da-e052-4760-b06d-104bf0b11e91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:58:19.698656 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:58:19.698656 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:58:19.698656 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:58:19.698656 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:58:19.698656 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:58:19.698656 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:58:19.698656 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:58:19.698656 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:58:19.698656 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:58:19.698656 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:58:19.698656 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:58:19.698656 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 06 16:58:19.698656 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 06 16:58:19.698656 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 06 16:58:19.699720 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 06 16:58:19.699720 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:58:19.699720 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:58:19.699720 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:58:19.699720 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:58:19.699720 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:58:19.699720 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:58:19.699720 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:58:19.699720 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application with Lock( Jul 06 16:58:19.699720 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:58:19.699720 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:58:19.699720 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:58:19.699720 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:58:19.699720 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:58:19.699720 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:58:19.699720 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:58:19.699720 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:58:19.700827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 06 16:58:19.700827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 06 16:58:19.700827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 06 16:58:19.700827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 06 16:58:19.700827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 06 16:58:19.700827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 06 16:58:19.700827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 06 16:58:19.700827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: ERROR keystone.server.flask.application Jul 06 16:58:19.700827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 656/1285] 10.222.0.40 () {56 vars in 1111 bytes} [Wed Jul 6 16:58:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:58:19.705675 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff58353-b323-4c62-9701-be5a01e35418 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:19.755662 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:19.803917 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff58353-b323-4c62-9701-be5a01e35418 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:19.837171 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990f063e-1481-452d-94fd-71caee5f4f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:19.837833 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f063e-1481-452d-94fd-71caee5f4f04 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:19.838123 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f063e-1481-452d-94fd-71caee5f4f04 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:19.838404 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f063e-1481-452d-94fd-71caee5f4f04 None admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:19.839183 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990f063e-1481-452d-94fd-71caee5f4f04 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:58:19.839495 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990f063e-1481-452d-94fd-71caee5f4f04 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:19.840962 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990f063e-1481-452d-94fd-71caee5f4f04 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:19.853304 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 630/1286] 10.222.0.40 () {56 vars in 1148 bytes} [Wed Jul 6 16:58:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:58:19.859283 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76397da-e052-4760-b06d-104bf0b11e91 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:19.955435 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76397da-e052-4760-b06d-104bf0b11e91 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:19.990964 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521df418-79eb-4dfc-ad3c-883027d6ba03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:19.991629 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521df418-79eb-4dfc-ad3c-883027d6ba03 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:19.991947 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521df418-79eb-4dfc-ad3c-883027d6ba03 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:19.992228 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521df418-79eb-4dfc-ad3c-883027d6ba03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:19.993063 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521df418-79eb-4dfc-ad3c-883027d6ba03 None admin] RBAC: Adding query filter params (role.id=b41f8a7a58e64f749f2099ca45cc04a7, scope.project.id=7938b0346d4240ab89774327d09a89ae, user.id=76ca877bee9041d28ad41e45ef3dd18c) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:58:19.993380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521df418-79eb-4dfc-ad3c-883027d6ba03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:19.994818 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521df418-79eb-4dfc-ad3c-883027d6ba03 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:20.000705 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 657/1287] 10.222.0.40 () {56 vars in 1374 bytes} [Wed Jul 6 16:58:19 2022] GET /identity/v3/role_assignments?user.id=76ca877bee9041d28ad41e45ef3dd18c&scope.project.id=7938b0346d4240ab89774327d09a89ae&role.id=b41f8a7a58e64f749f2099ca45cc04a7 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 16:58:22.059007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf631692-cbf8-4399-926d-a47635c53073 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:22.059549 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf631692-cbf8-4399-926d-a47635c53073 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:22.059549 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf631692-cbf8-4399-926d-a47635c53073 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:22.060881 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 631/1288] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:58:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:58:22.065840 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcfde10-9f14-444d-867f-75064d8e56f7 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:22.066286 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcfde10-9f14-444d-867f-75064d8e56f7 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:22.068447 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcfde10-9f14-444d-867f-75064d8e56f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:22.083578 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-8dcfde10-9f14-444d-867f-75064d8e56f7 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:58:22.084844 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-8dcfde10-9f14-444d-867f-75064d8e56f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:58:22.113492 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:22.150930 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 658/1289] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:58:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:58:22.172549 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf631692-cbf8-4399-926d-a47635c53073 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:22.272070 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf631692-cbf8-4399-926d-a47635c53073 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:22.282284 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:22.305889 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487afa79-67bf-45e8-be45-856ae44b72d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:22.306681 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487afa79-67bf-45e8-be45-856ae44b72d7 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:22.306965 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487afa79-67bf-45e8-be45-856ae44b72d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:22.307240 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487afa79-67bf-45e8-be45-856ae44b72d7 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:22.307922 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487afa79-67bf-45e8-be45-856ae44b72d7 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:22.314959 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487afa79-67bf-45e8-be45-856ae44b72d7 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:22.320952 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-487afa79-67bf-45e8-be45-856ae44b72d7 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 06 16:58:22.320952 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:58:22.320952 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 16:58:22.320952 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 16:58:22.320952 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 16:58:22.320952 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 16:58:22.320952 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 16:58:22.320952 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:58:22.320952 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 06 16:58:22.320952 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:58:22.320952 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 16:58:22.320952 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:58:22.320952 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:58:22.320952 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:58:22.320952 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:58:22.320952 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:58:22.320952 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:58:22.322151 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:58:22.322151 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:58:22.322151 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:58:22.322151 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:58:22.322151 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:58:22.322151 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:58:22.322151 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 16:58:22.322151 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 16:58:22.322151 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 16:58:22.322151 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 16:58:22.322151 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:58:22.322151 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:58:22.322151 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:58:22.322151 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:58:22.322151 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:58:22.322151 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:58:22.323037 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:58:22.323037 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:58:22.323037 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:58:22.323037 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:58:22.323037 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:58:22.323037 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:58:22.323037 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:58:22.323037 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:58:22.323037 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:58:22.323037 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:58:22.323037 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 16:58:22.323037 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:58:22.323037 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 06 16:58:22.323037 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:58:22.323037 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 632/1290] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Jul 6 16:58:22 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:58:22.328051 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcfde10-9f14-444d-867f-75064d8e56f7 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:22.425659 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcfde10-9f14-444d-867f-75064d8e56f7 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:22.459306 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f66743-1fc6-421d-8276-0dddf1cec79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:22.460041 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f66743-1fc6-421d-8276-0dddf1cec79b None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:22.460325 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f66743-1fc6-421d-8276-0dddf1cec79b None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:22.460597 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f66743-1fc6-421d-8276-0dddf1cec79b None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:22.461299 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f66743-1fc6-421d-8276-0dddf1cec79b None admin] RBAC: Adding query filter params (name=placement) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:58:22.461603 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f66743-1fc6-421d-8276-0dddf1cec79b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:22.462956 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f66743-1fc6-421d-8276-0dddf1cec79b None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:22.467806 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 659/1291] 10.222.0.40 () {56 vars in 1116 bytes} [Wed Jul 6 16:58:22 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:58:22.473648 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487afa79-67bf-45e8-be45-856ae44b72d7 None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:22.570740 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487afa79-67bf-45e8-be45-856ae44b72d7 None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:22.603031 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0171770a-f093-446e-a7c9-6e30d420d307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:22.603666 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171770a-f093-446e-a7c9-6e30d420d307 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:22.604029 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171770a-f093-446e-a7c9-6e30d420d307 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:22.604304 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0171770a-f093-446e-a7c9-6e30d420d307 None admin] PATH_INFO: `/v3/services` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:22.604991 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0171770a-f093-446e-a7c9-6e30d420d307 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:58:22.605293 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0171770a-f093-446e-a7c9-6e30d420d307 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:22.606652 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0171770a-f093-446e-a7c9-6e30d420d307 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:22.611038 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 633/1292] 10.222.0.40 () {56 vars in 1116 bytes} [Wed Jul 6 16:58:22 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:58:24.598017 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7590f8a-5265-4471-9f64-2c455fa19fe3 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:24.598557 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7590f8a-5265-4471-9f64-2c455fa19fe3 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:24.598557 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7590f8a-5265-4471-9f64-2c455fa19fe3 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:24.600558 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 660/1293] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:58:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:58:24.605664 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0d3b9-4458-494f-8703-063f4fa187a5 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:24.606106 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0d3b9-4458-494f-8703-063f4fa187a5 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:24.606232 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c0d3b9-4458-494f-8703-063f4fa187a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:24.622730 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-02c0d3b9-4458-494f-8703-063f4fa187a5 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:58:24.623932 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-02c0d3b9-4458-494f-8703-063f4fa187a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:58:24.654974 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:24.691972 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 634/1294] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:58:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:58:24.716067 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7590f8a-5265-4471-9f64-2c455fa19fe3 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:24.746674 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:24.814991 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7590f8a-5265-4471-9f64-2c455fa19fe3 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:24.849226 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d35c26-ad1c-434b-aa8a-d387a4cb4d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:24.849828 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d35c26-ad1c-434b-aa8a-d387a4cb4d50 None admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:24.850115 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d35c26-ad1c-434b-aa8a-d387a4cb4d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:24.850392 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d35c26-ad1c-434b-aa8a-d387a4cb4d50 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:24.851179 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d35c26-ad1c-434b-aa8a-d387a4cb4d50 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:24.852575 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d35c26-ad1c-434b-aa8a-d387a4cb4d50 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:24.938253 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 661/1295] 10.222.0.40 () {60 vars in 1141 bytes} [Wed Jul 6 16:58:24 2022] POST /identity/v3/services => generated 246 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 16:58:26.969992 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326b8e5-34dc-4af7-a55f-1df8e4df3e89 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:26.970542 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326b8e5-34dc-4af7-a55f-1df8e4df3e89 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:26.970542 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326b8e5-34dc-4af7-a55f-1df8e4df3e89 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:26.971959 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 635/1296] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:58:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:58:26.977027 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55de51-f0b5-41c9-8d9a-cb5459436bde None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:26.977414 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55de51-f0b5-41c9-8d9a-cb5459436bde None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:26.977583 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55de51-f0b5-41c9-8d9a-cb5459436bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:26.994765 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-1b55de51-f0b5-41c9-8d9a-cb5459436bde None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:58:26.995888 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-1b55de51-f0b5-41c9-8d9a-cb5459436bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:58:27.068821 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 662/1297] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:58:26 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:58:27.092225 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e326b8e5-34dc-4af7-a55f-1df8e4df3e89 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:27.191581 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e326b8e5-34dc-4af7-a55f-1df8e4df3e89 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:27.225889 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba5321e-b1c8-4f67-b6af-77404a41123b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:27.226676 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5321e-b1c8-4f67-b6af-77404a41123b None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:27.226957 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5321e-b1c8-4f67-b6af-77404a41123b None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:27.227339 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba5321e-b1c8-4f67-b6af-77404a41123b None admin] PATH_INFO: `/v3/services/placement` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:27.228016 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba5321e-b1c8-4f67-b6af-77404a41123b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:27.229381 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba5321e-b1c8-4f67-b6af-77404a41123b None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:27.241187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-6ba5321e-b1c8-4f67-b6af-77404a41123b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 06 16:58:27.241187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:58:27.241187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 16:58:27.241187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 16:58:27.241187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 16:58:27.241187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 16:58:27.241187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 16:58:27.241187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:58:27.241187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 06 16:58:27.241187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:58:27.241187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 16:58:27.241187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:58:27.241187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:58:27.241187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:58:27.241187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:58:27.241187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:58:27.241187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:58:27.242289 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:58:27.242289 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:58:27.242289 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:58:27.242289 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:58:27.242289 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:58:27.242289 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:58:27.242289 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 16:58:27.242289 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 16:58:27.242289 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 16:58:27.242289 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 16:58:27.242289 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:58:27.242289 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:58:27.242289 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:58:27.242289 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:58:27.242289 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:58:27.242289 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:58:27.243150 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:58:27.243150 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:58:27.243150 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:58:27.243150 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:58:27.243150 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:58:27.243150 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:58:27.243150 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:58:27.243150 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:58:27.243150 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:58:27.243150 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:58:27.243150 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 16:58:27.243150 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:58:27.243150 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 06 16:58:27.243150 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:58:27.243150 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 636/1298] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Jul 6 16:58:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:58:27.248553 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55de51-f0b5-41c9-8d9a-cb5459436bde None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:27.266765 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:27.343994 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55de51-f0b5-41c9-8d9a-cb5459436bde None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:27.378054 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6108b11-574c-4dcd-bb84-542667a20251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:27.378716 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6108b11-574c-4dcd-bb84-542667a20251 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:27.379002 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6108b11-574c-4dcd-bb84-542667a20251 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:27.379279 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6108b11-574c-4dcd-bb84-542667a20251 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:27.380543 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6108b11-574c-4dcd-bb84-542667a20251 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:58:27.380862 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6108b11-574c-4dcd-bb84-542667a20251 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:27.382224 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6108b11-574c-4dcd-bb84-542667a20251 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:27.387386 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 663/1299] 10.222.0.40 () {56 vars in 1116 bytes} [Wed Jul 6 16:58:27 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:58:27.393371 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba5321e-b1c8-4f67-b6af-77404a41123b None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:27.424309 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:27.491527 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba5321e-b1c8-4f67-b6af-77404a41123b None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:27.529286 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54196f88-483c-4065-ad04-21d292298105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:27.529879 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54196f88-483c-4065-ad04-21d292298105 None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:27.530161 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54196f88-483c-4065-ad04-21d292298105 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:27.530435 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54196f88-483c-4065-ad04-21d292298105 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:27.531216 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54196f88-483c-4065-ad04-21d292298105 None admin] RBAC: Adding query filter params (interface=public, service_id=088df5ac893d43ba8349ab8bf8b75c47, region_id=RegionOne) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:58:27.531526 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54196f88-483c-4065-ad04-21d292298105 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:27.532930 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54196f88-483c-4065-ad04-21d292298105 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:27.537836 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 637/1300] 10.222.0.40 () {56 vars in 1251 bytes} [Wed Jul 6 16:58:27 2022] GET /identity/v3/endpoints?service_id=088df5ac893d43ba8349ab8bf8b75c47&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:58:27.543504 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6108b11-574c-4dcd-bb84-542667a20251 None admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:27.624885 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:27.642789 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6108b11-574c-4dcd-bb84-542667a20251 None admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:27.675146 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d47d7e5-b7d0-48b7-9166-aa14b4b85e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:27.675835 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47d7e5-b7d0-48b7-9166-aa14b4b85e59 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:27.676124 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47d7e5-b7d0-48b7-9166-aa14b4b85e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:27.676419 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47d7e5-b7d0-48b7-9166-aa14b4b85e59 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:27.677656 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d47d7e5-b7d0-48b7-9166-aa14b4b85e59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:27.679670 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d47d7e5-b7d0-48b7-9166-aa14b4b85e59 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:27.686821 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 664/1301] 10.222.0.40 () {56 vars in 1087 bytes} [Wed Jul 6 16:58:27 2022] GET /identity/v3/services => generated 2191 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 16:58:29.688324 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf1d6f-5d77-471f-81a7-753436d47187 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:29.688905 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf1d6f-5d77-471f-81a7-753436d47187 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:29.688905 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf1d6f-5d77-471f-81a7-753436d47187 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:29.690230 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 638/1302] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 16:58:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 16:58:29.695796 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b128cd1-63f2-43ac-804e-6e32f62eed14 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:29.696229 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b128cd1-63f2-43ac-804e-6e32f62eed14 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:29.696401 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b128cd1-63f2-43ac-804e-6e32f62eed14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:29.713474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-0b128cd1-63f2-43ac-804e-6e32f62eed14 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 16:58:29.714683 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-0b128cd1-63f2-43ac-804e-6e32f62eed14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 16:58:29.781830 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 665/1303] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 16:58:29 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 06 16:58:29.804509 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bf1d6f-5d77-471f-81a7-753436d47187 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:29.896470 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:29.903326 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bf1d6f-5d77-471f-81a7-753436d47187 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:29.935860 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21934be-6c97-4471-a261-f3fd2944ab0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:29.936648 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21934be-6c97-4471-a261-f3fd2944ab0e None admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:29.936929 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21934be-6c97-4471-a261-f3fd2944ab0e None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:29.937202 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21934be-6c97-4471-a261-f3fd2944ab0e None admin] PATH_INFO: `/v3/services/placement` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:29.937840 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21934be-6c97-4471-a261-f3fd2944ab0e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:29.939234 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21934be-6c97-4471-a261-f3fd2944ab0e None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:29.951193 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application [None req-e21934be-6c97-4471-a261-f3fd2944ab0e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 06 16:58:29.951193 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:58:29.951193 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 06 16:58:29.951193 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 06 16:58:29.951193 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 06 16:58:29.951193 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 06 16:58:29.951193 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 06 16:58:29.951193 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:58:29.951193 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 06 16:58:29.951193 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:58:29.951193 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 06 16:58:29.951193 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:58:29.951193 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 06 16:58:29.951193 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 06 16:58:29.951193 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 06 16:58:29.951193 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 06 16:58:29.951193 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 06 16:58:29.958450 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 06 16:58:29.958450 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 06 16:58:29.958450 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 06 16:58:29.958450 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 06 16:58:29.958450 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 06 16:58:29.958450 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 06 16:58:29.958450 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 06 16:58:29.958450 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 06 16:58:29.958450 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 06 16:58:29.958450 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 06 16:58:29.958450 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 06 16:58:29.958450 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 06 16:58:29.958450 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 06 16:58:29.958450 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 06 16:58:29.958450 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 06 16:58:29.958450 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.get_or_create( Jul 06 16:58:29.959540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 06 16:58:29.959540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application with Lock( Jul 06 16:58:29.959540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 06 16:58:29.959540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self._enter() Jul 06 16:58:29.959540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 06 16:58:29.959540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 06 16:58:29.959540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 06 16:58:29.959540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application return self.creator() Jul 06 16:58:29.959540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 06 16:58:29.959540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application created_value = creator( Jul 06 16:58:29.959540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 06 16:58:29.959540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 06 16:58:29.959540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 06 16:58:29.959540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: ERROR keystone.server.flask.application Jul 06 16:58:29.959540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 639/1304] 10.222.0.40 () {56 vars in 1117 bytes} [Wed Jul 6 16:58:29 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 06 16:58:29.960573 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b128cd1-63f2-43ac-804e-6e32f62eed14 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:30.033370 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:30.055944 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b128cd1-63f2-43ac-804e-6e32f62eed14 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:30.088788 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e13e17-a334-42a6-851c-eb788b993039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:30.089451 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e13e17-a334-42a6-851c-eb788b993039 None admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:30.089738 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e13e17-a334-42a6-851c-eb788b993039 None admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:30.090017 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e13e17-a334-42a6-851c-eb788b993039 None admin] PATH_INFO: `/v3/services` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:30.090770 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e13e17-a334-42a6-851c-eb788b993039 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 16:58:30.091081 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e13e17-a334-42a6-851c-eb788b993039 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:30.092473 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e13e17-a334-42a6-851c-eb788b993039 None admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:30.097544 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 666/1305] 10.222.0.40 () {56 vars in 1116 bytes} [Wed Jul 6 16:58:29 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 16:58:30.103887 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21934be-6c97-4471-a261-f3fd2944ab0e None admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 16:58:30.200842 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21934be-6c97-4471-a261-f3fd2944ab0e None admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 16:58:30.233728 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7652ec0c-2147-43b8-bcd2-d4051e9aeba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 16:58:30.234380 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652ec0c-2147-43b8-bcd2-d4051e9aeba5 None admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 16:58:30.234665 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652ec0c-2147-43b8-bcd2-d4051e9aeba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 16:58:30.234942 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7652ec0c-2147-43b8-bcd2-d4051e9aeba5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 16:58:30.235667 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7652ec0c-2147-43b8-bcd2-d4051e9aeba5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 16:58:30.237093 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7652ec0c-2147-43b8-bcd2-d4051e9aeba5 None admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 16:58:30.331786 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 640/1306] 10.222.0.40 () {60 vars in 1144 bytes} [Wed Jul 6 16:58:30 2022] POST /identity/v3/endpoints => generated 331 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 16:58:40.094614 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=30 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 16:58:40.327255 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:04:57.029270 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ec976-70be-4ff1-bfaf-53724b3225fd None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:04:57.034501 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ec976-70be-4ff1-bfaf-53724b3225fd None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:04:57.034501 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ec976-70be-4ff1-bfaf-53724b3225fd None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:04:57.037562 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e852936-c778-48f0-9898-754de60b640b None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:04:57.038042 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e852936-c778-48f0-9898-754de60b640b None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:04:57.038134 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e852936-c778-48f0-9898-754de60b640b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:04:57.041635 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 667/1307] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 17:04:57 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:04:57.127753 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-6e852936-c778-48f0-9898-754de60b640b None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:04:57.128928 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-6e852936-c778-48f0-9898-754de60b640b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:04:57.264013 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 641/1308] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 17:04:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:04:57.296548 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687cc98-eb9c-47b6-910e-6650d5ccbf71 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:04:57.297191 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687cc98-eb9c-47b6-910e-6650d5ccbf71 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:04:57.297719 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6687cc98-eb9c-47b6-910e-6650d5ccbf71 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:04:57.299625 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 668/1309] 10.222.0.40 () {54 vars in 964 bytes} [Wed Jul 6 17:04:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:04:57.305449 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c1322-de51-4072-a45b-a89d3750a0c2 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:04:57.305748 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c1322-de51-4072-a45b-a89d3750a0c2 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:04:57.306037 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388c1322-de51-4072-a45b-a89d3750a0c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:04:57.409180 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-388c1322-de51-4072-a45b-a89d3750a0c2 None None] MFA Rules not processed for user `5f479458a68b4daf9f3a7ae53d6c3763`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:04:57.411849 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-388c1322-de51-4072-a45b-a89d3750a0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:04:57.455502 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:04:57.553693 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 642/1310] 10.222.0.40 () {58 vars in 1061 bytes} [Wed Jul 6 17:04:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:04:57.563295 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6687cc98-eb9c-47b6-910e-6650d5ccbf71 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:04:57.565237 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-6687cc98-eb9c-47b6-910e-6650d5ccbf71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:04:57.739892 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6687cc98-eb9c-47b6-910e-6650d5ccbf71 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:04:57.825490 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12af697-c839-4059-a123-02a5d18eee7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:04:57.826099 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12af697-c839-4059-a123-02a5d18eee7f service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:04:57.826381 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12af697-c839-4059-a123-02a5d18eee7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:04:57.826659 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12af697-c839-4059-a123-02a5d18eee7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:04:57.828402 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-e12af697-c839-4059-a123-02a5d18eee7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:04:57.889113 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:04:57.946345 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12af697-c839-4059-a123-02a5d18eee7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:04:57.947777 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12af697-c839-4059-a123-02a5d18eee7f service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:04:57.967532 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 669/1311] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:04:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:05:00.380248 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ecba88-0e93-4413-b16d-4a114dcfdc4a None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:00.380790 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ecba88-0e93-4413-b16d-4a114dcfdc4a None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:00.380790 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ecba88-0e93-4413-b16d-4a114dcfdc4a None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:00.382408 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 643/1312] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 17:05:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:05:00.388183 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133ec55-993f-4539-9c41-b762c4832ccb None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:00.388756 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133ec55-993f-4539-9c41-b762c4832ccb None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:00.388756 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c133ec55-993f-4539-9c41-b762c4832ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:00.474187 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-c133ec55-993f-4539-9c41-b762c4832ccb None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:05:00.475404 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c133ec55-993f-4539-9c41-b762c4832ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:05:00.576203 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 670/1313] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 17:05:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:05:00.615115 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ecba88-0e93-4413-b16d-4a114dcfdc4a None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:05:00.616736 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-06ecba88-0e93-4413-b16d-4a114dcfdc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:05:00.734682 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ecba88-0e93-4413-b16d-4a114dcfdc4a None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:05:00.805194 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:05:00.807796 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a07f3f6-da39-46cd-993e-46c48f964c7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:05:00.808457 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07f3f6-da39-46cd-993e-46c48f964c7a service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:00.808741 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07f3f6-da39-46cd-993e-46c48f964c7a service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:00.809017 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07f3f6-da39-46cd-993e-46c48f964c7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:00.810650 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-5a07f3f6-da39-46cd-993e-46c48f964c7a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:05:00.920066 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a07f3f6-da39-46cd-993e-46c48f964c7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:05:00.921511 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a07f3f6-da39-46cd-993e-46c48f964c7a service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:05:00.941753 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 644/1314] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:05:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:05:03.543367 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa280e1f-812b-419a-87b5-b205cd304eaf None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:03.543968 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa280e1f-812b-419a-87b5-b205cd304eaf None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:03.543968 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa280e1f-812b-419a-87b5-b205cd304eaf None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:03.545324 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 671/1315] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 17:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:05:03.550332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d69454-9c71-4aea-8301-7f4a66820985 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:03.550740 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d69454-9c71-4aea-8301-7f4a66820985 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:03.550889 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d69454-9c71-4aea-8301-7f4a66820985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:03.617362 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-27d69454-9c71-4aea-8301-7f4a66820985 None None] MFA Rules not processed for user `8ad29aae43d24936a5dbec7f54883329`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:05:03.618569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-27d69454-9c71-4aea-8301-7f4a66820985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:05:03.669523 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:05:03.713729 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 645/1316] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 17:05:03 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:05:03.752837 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa280e1f-812b-419a-87b5-b205cd304eaf None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:05:03.754409 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-aa280e1f-812b-419a-87b5-b205cd304eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:05:03.764206 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:05:03.860260 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa280e1f-812b-419a-87b5-b205cd304eaf None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:05:03.933471 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a80018-de55-4fd8-abf4-3a4110f1f279 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:05:03.934113 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a80018-de55-4fd8-abf4-3a4110f1f279 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:03.934393 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a80018-de55-4fd8-abf4-3a4110f1f279 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:03.934666 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a80018-de55-4fd8-abf4-3a4110f1f279 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:03.936399 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-08a80018-de55-4fd8-abf4-3a4110f1f279 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:05:03.999608 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:05:04.045410 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a80018-de55-4fd8-abf4-3a4110f1f279 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:05:04.046844 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a80018-de55-4fd8-abf4-3a4110f1f279 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:05:04.065858 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 672/1317] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:05:03 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:05:06.891937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dac797-0ee6-43d2-a95f-f859f628fb93 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:06.892494 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dac797-0ee6-43d2-a95f-f859f628fb93 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:06.892897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dac797-0ee6-43d2-a95f-f859f628fb93 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:06.894714 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 646/1318] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 17:05:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:05:06.900247 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2e223-e407-465f-9911-7e8e68bfbfb8 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:06.900667 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2e223-e407-465f-9911-7e8e68bfbfb8 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:06.901070 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e2e223-e407-465f-9911-7e8e68bfbfb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:06.964240 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-d2e2e223-e407-465f-9911-7e8e68bfbfb8 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:05:06.965459 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d2e2e223-e407-465f-9911-7e8e68bfbfb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:05:07.060138 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 673/1319] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 17:05:06 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:05:07.097808 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dac797-0ee6-43d2-a95f-f859f628fb93 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:05:07.099314 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c8dac797-0ee6-43d2-a95f-f859f628fb93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:05:07.200215 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dac797-0ee6-43d2-a95f-f859f628fb93 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:05:07.269837 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae74d30d-99eb-4ec0-8c5f-fe284053b0aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:05:07.270472 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74d30d-99eb-4ec0-8c5f-fe284053b0aa service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:07.270755 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74d30d-99eb-4ec0-8c5f-fe284053b0aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:07.271029 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae74d30d-99eb-4ec0-8c5f-fe284053b0aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:07.272653 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ae74d30d-99eb-4ec0-8c5f-fe284053b0aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:05:07.280378 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:05:07.378805 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae74d30d-99eb-4ec0-8c5f-fe284053b0aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:05:07.380193 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae74d30d-99eb-4ec0-8c5f-fe284053b0aa service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:05:07.398680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 647/1320] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:05:07 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:05:10.610713 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83101a1-2bf1-4159-a901-43faa139e212 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:10.611766 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83101a1-2bf1-4159-a901-43faa139e212 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:10.611860 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83101a1-2bf1-4159-a901-43faa139e212 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:10.613189 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 674/1321] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 17:05:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:05:10.618359 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b4a847-6cb6-48d4-aa9f-85cbf684703c None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:10.618800 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b4a847-6cb6-48d4-aa9f-85cbf684703c None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:10.618930 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b4a847-6cb6-48d4-aa9f-85cbf684703c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:10.683647 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-48b4a847-6cb6-48d4-aa9f-85cbf684703c None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:05:10.684867 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-48b4a847-6cb6-48d4-aa9f-85cbf684703c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:05:10.744584 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:05:10.783509 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 648/1322] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 17:05:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:05:10.819468 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83101a1-2bf1-4159-a901-43faa139e212 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:05:10.837615 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c83101a1-2bf1-4159-a901-43faa139e212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:05:10.846509 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:05:10.935319 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83101a1-2bf1-4159-a901-43faa139e212 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:05:11.015460 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabe6a96-8a48-47f7-ba28-f2d500bf3d87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:05:11.016154 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe6a96-8a48-47f7-ba28-f2d500bf3d87 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:11.016442 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe6a96-8a48-47f7-ba28-f2d500bf3d87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:11.016719 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabe6a96-8a48-47f7-ba28-f2d500bf3d87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:11.018443 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-dabe6a96-8a48-47f7-ba28-f2d500bf3d87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:05:11.090083 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:05:11.128162 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabe6a96-8a48-47f7-ba28-f2d500bf3d87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:05:11.135077 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabe6a96-8a48-47f7-ba28-f2d500bf3d87 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:05:11.147570 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 675/1323] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:05:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:05:13.226978 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3be35-3377-4d35-8e2e-ff61c512bc03 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:13.227534 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3be35-3377-4d35-8e2e-ff61c512bc03 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:13.227534 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3be35-3377-4d35-8e2e-ff61c512bc03 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:13.228932 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 649/1324] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 17:05:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:05:13.234287 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d1f209-b8e5-43e4-bc99-519499f78885 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:13.234717 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d1f209-b8e5-43e4-bc99-519499f78885 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:13.234977 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d1f209-b8e5-43e4-bc99-519499f78885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:13.300476 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-90d1f209-b8e5-43e4-bc99-519499f78885 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:05:13.301586 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-90d1f209-b8e5-43e4-bc99-519499f78885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:05:13.397655 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 676/1325] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 17:05:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:05:13.432805 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da3be35-3377-4d35-8e2e-ff61c512bc03 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:05:13.434321 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-9da3be35-3377-4d35-8e2e-ff61c512bc03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:05:13.535393 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da3be35-3377-4d35-8e2e-ff61c512bc03 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:05:13.607714 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc947660-8b09-46e9-a520-a1482539a173 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:05:13.608310 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc947660-8b09-46e9-a520-a1482539a173 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:13.608592 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc947660-8b09-46e9-a520-a1482539a173 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:13.608868 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc947660-8b09-46e9-a520-a1482539a173 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:13.610996 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-cc947660-8b09-46e9-a520-a1482539a173 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:05:13.628058 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:05:13.727699 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc947660-8b09-46e9-a520-a1482539a173 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:05:13.729260 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc947660-8b09-46e9-a520-a1482539a173 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:05:13.748587 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 650/1326] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:05:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:05:15.889384 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc36ffe-da0d-4973-951c-33450bbcb984 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:15.889969 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc36ffe-da0d-4973-951c-33450bbcb984 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:15.890063 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc36ffe-da0d-4973-951c-33450bbcb984 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:15.892190 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 677/1327] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 17:05:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:05:15.898295 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29bef30-0ddf-4dff-b342-11b997cf175f None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:15.898855 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29bef30-0ddf-4dff-b342-11b997cf175f None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:15.898964 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29bef30-0ddf-4dff-b342-11b997cf175f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:15.963135 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-f29bef30-0ddf-4dff-b342-11b997cf175f None None] MFA Rules not processed for user `8ad29aae43d24936a5dbec7f54883329`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:05:15.964378 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-f29bef30-0ddf-4dff-b342-11b997cf175f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:05:16.030762 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:05:16.063038 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 651/1328] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 17:05:15 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:05:16.116406 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae43c0-96f2-4649-a29e-e54198960e9b None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:16.116810 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae43c0-96f2-4649-a29e-e54198960e9b None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:16.117187 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ae43c0-96f2-4649-a29e-e54198960e9b None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:16.118563 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 678/1329] 10.222.0.40 () {54 vars in 964 bytes} [Wed Jul 6 17:05:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:05:16.124487 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a0df6a-b152-4fdc-81fb-d208c8005646 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:16.124779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a0df6a-b152-4fdc-81fb-d208c8005646 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:16.125059 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a0df6a-b152-4fdc-81fb-d208c8005646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:16.211206 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-28a0df6a-b152-4fdc-81fb-d208c8005646 None None] MFA Rules not processed for user `5f479458a68b4daf9f3a7ae53d6c3763`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:05:16.212442 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-28a0df6a-b152-4fdc-81fb-d208c8005646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:05:16.288393 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 652/1330] 10.222.0.40 () {58 vars in 1061 bytes} [Wed Jul 6 17:05:16 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:05:16.297286 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae43c0-96f2-4649-a29e-e54198960e9b None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:05:16.299532 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-a3ae43c0-96f2-4649-a29e-e54198960e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:05:16.329995 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:05:16.405235 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ae43c0-96f2-4649-a29e-e54198960e9b None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:05:16.478703 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14beb3e-d13e-4eff-a038-826885ab6635 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:05:16.479358 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14beb3e-d13e-4eff-a038-826885ab6635 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:16.479643 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14beb3e-d13e-4eff-a038-826885ab6635 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:16.479952 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14beb3e-d13e-4eff-a038-826885ab6635 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:16.481601 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-a14beb3e-d13e-4eff-a038-826885ab6635 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:05:16.572720 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:05:16.606838 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14beb3e-d13e-4eff-a038-826885ab6635 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:05:16.608341 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14beb3e-d13e-4eff-a038-826885ab6635 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:05:16.629855 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 679/1331] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:05:16 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:05:19.554763 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0e3de-c4fc-401d-b220-c44640243907 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:19.555369 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0e3de-c4fc-401d-b220-c44640243907 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:19.555369 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb0e3de-c4fc-401d-b220-c44640243907 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:19.556746 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 653/1332] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 17:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:05:19.561838 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c37ad5-a20e-4dbf-a7ea-e14d3a2c96df None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:19.562286 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c37ad5-a20e-4dbf-a7ea-e14d3a2c96df None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:19.562400 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c37ad5-a20e-4dbf-a7ea-e14d3a2c96df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:19.625497 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-e0c37ad5-a20e-4dbf-a7ea-e14d3a2c96df None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:05:19.627210 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-e0c37ad5-a20e-4dbf-a7ea-e14d3a2c96df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:05:19.722129 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 680/1333] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 17:05:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:05:19.758201 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb0e3de-c4fc-401d-b220-c44640243907 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:05:19.759746 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-cbb0e3de-c4fc-401d-b220-c44640243907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:05:19.828494 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:05:19.862267 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb0e3de-c4fc-401d-b220-c44640243907 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:05:19.935989 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27ed93-8502-49c4-9ea4-289375279645 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:05:19.936641 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27ed93-8502-49c4-9ea4-289375279645 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:19.936922 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27ed93-8502-49c4-9ea4-289375279645 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:19.937202 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27ed93-8502-49c4-9ea4-289375279645 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:19.938831 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4a27ed93-8502-49c4-9ea4-289375279645 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:05:20.050984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a27ed93-8502-49c4-9ea4-289375279645 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:05:20.052461 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a27ed93-8502-49c4-9ea4-289375279645 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:05:20.062306 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:05:20.072430 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 654/1334] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:05:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:05:22.134740 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f56d67-8222-41d1-aca8-728ee825dcca None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:22.135262 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f56d67-8222-41d1-aca8-728ee825dcca None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:22.135262 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f56d67-8222-41d1-aca8-728ee825dcca None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:22.136613 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 681/1335] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 17:05:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:05:22.141709 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a307ffdd-ee51-48c9-bf0b-b7c5e66caa82 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:22.142156 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a307ffdd-ee51-48c9-bf0b-b7c5e66caa82 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:22.142252 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a307ffdd-ee51-48c9-bf0b-b7c5e66caa82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:22.205293 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-a307ffdd-ee51-48c9-bf0b-b7c5e66caa82 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:05:22.206506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-a307ffdd-ee51-48c9-bf0b-b7c5e66caa82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:05:22.306006 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 655/1336] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 17:05:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:05:22.345622 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f56d67-8222-41d1-aca8-728ee825dcca None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:05:22.347175 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-39f56d67-8222-41d1-aca8-728ee825dcca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:05:22.387157 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:05:22.449982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f56d67-8222-41d1-aca8-728ee825dcca None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:05:22.522261 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c08d62b-5b88-4e23-8bb2-3fb98559e0d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:05:22.522908 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08d62b-5b88-4e23-8bb2-3fb98559e0d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:22.523198 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08d62b-5b88-4e23-8bb2-3fb98559e0d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:22.523477 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c08d62b-5b88-4e23-8bb2-3fb98559e0d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:22.525142 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-8c08d62b-5b88-4e23-8bb2-3fb98559e0d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:05:22.606891 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:05:22.632316 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c08d62b-5b88-4e23-8bb2-3fb98559e0d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:05:22.633751 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c08d62b-5b88-4e23-8bb2-3fb98559e0d9 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:05:22.652310 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 682/1337] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:05:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:05:25.369915 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666e0c8-736a-46f6-a2a7-73887c642880 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:25.370452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666e0c8-736a-46f6-a2a7-73887c642880 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:25.370452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666e0c8-736a-46f6-a2a7-73887c642880 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:25.371802 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 656/1338] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 17:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:05:25.377497 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a896cd3-cb3f-4b1f-8798-c12be683813c None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:25.377896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a896cd3-cb3f-4b1f-8798-c12be683813c None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:25.378091 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a896cd3-cb3f-4b1f-8798-c12be683813c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:25.442299 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-7a896cd3-cb3f-4b1f-8798-c12be683813c None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:05:25.443537 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-7a896cd3-cb3f-4b1f-8798-c12be683813c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:05:25.539236 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 683/1339] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 17:05:25 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:05:25.574820 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3666e0c8-736a-46f6-a2a7-73887c642880 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:05:25.576394 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-3666e0c8-736a-46f6-a2a7-73887c642880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:05:25.655036 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:05:25.684051 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3666e0c8-736a-46f6-a2a7-73887c642880 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:05:25.762761 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57a0234-24f9-44e7-8e85-3d1095afc717 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:05:25.763417 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a0234-24f9-44e7-8e85-3d1095afc717 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:25.763706 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a0234-24f9-44e7-8e85-3d1095afc717 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:25.764022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57a0234-24f9-44e7-8e85-3d1095afc717 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:25.765666 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-b57a0234-24f9-44e7-8e85-3d1095afc717 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:05:25.875004 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57a0234-24f9-44e7-8e85-3d1095afc717 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:05:25.877118 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57a0234-24f9-44e7-8e85-3d1095afc717 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:05:25.895340 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 657/1340] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:05:25 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:05:30.221650 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af0292-6af8-482d-ab0d-d0681d8687c7 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:30.222295 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af0292-6af8-482d-ab0d-d0681d8687c7 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:30.222295 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1af0292-6af8-482d-ab0d-d0681d8687c7 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:30.223965 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 684/1341] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 17:05:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:05:30.229235 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ae1d1-d8c9-472e-a872-cf8a98d14085 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:30.229647 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ae1d1-d8c9-472e-a872-cf8a98d14085 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:30.229725 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ae1d1-d8c9-472e-a872-cf8a98d14085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:30.236415 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:05:30.304211 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-cf5ae1d1-d8c9-472e-a872-cf8a98d14085 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:05:30.305415 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-cf5ae1d1-d8c9-472e-a872-cf8a98d14085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:05:30.419665 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 658/1342] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 17:05:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:05:30.458923 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af0292-6af8-482d-ab0d-d0681d8687c7 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:05:30.461048 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c1af0292-6af8-482d-ab0d-d0681d8687c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:05:30.512235 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:05:30.557826 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1af0292-6af8-482d-ab0d-d0681d8687c7 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:05:30.561373 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb265f4-0e13-4786-a1c2-fc33a58a269e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:05:30.562025 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb265f4-0e13-4786-a1c2-fc33a58a269e service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:30.562308 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb265f4-0e13-4786-a1c2-fc33a58a269e service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:30.562581 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb265f4-0e13-4786-a1c2-fc33a58a269e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:30.571958 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb265f4-0e13-4786-a1c2-fc33a58a269e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:05:30.573381 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb265f4-0e13-4786-a1c2-fc33a58a269e service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:05:30.583241 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 685/1343] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:05:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:05:33.588632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a922c046-ad7c-4295-bca0-0bd250c2286f None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:33.589217 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a922c046-ad7c-4295-bca0-0bd250c2286f None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:33.589217 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a922c046-ad7c-4295-bca0-0bd250c2286f None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:33.590612 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 659/1344] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 17:05:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:05:33.596934 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98157d4e-f65c-4b8d-97eb-b8aaac1c251b None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:33.597516 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98157d4e-f65c-4b8d-97eb-b8aaac1c251b None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:33.597653 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98157d4e-f65c-4b8d-97eb-b8aaac1c251b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:33.627213 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-98157d4e-f65c-4b8d-97eb-b8aaac1c251b None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:05:33.628453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-98157d4e-f65c-4b8d-97eb-b8aaac1c251b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:05:33.720723 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 686/1345] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 17:05:33 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:05:33.758666 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a922c046-ad7c-4295-bca0-0bd250c2286f None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:05:33.769219 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a922c046-ad7c-4295-bca0-0bd250c2286f None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:05:33.773118 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c21ad27-9f69-41dc-bd4f-5ee4fa4bdbf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:05:33.773748 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c21ad27-9f69-41dc-bd4f-5ee4fa4bdbf7 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:33.774034 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c21ad27-9f69-41dc-bd4f-5ee4fa4bdbf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:33.774304 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c21ad27-9f69-41dc-bd4f-5ee4fa4bdbf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:33.784173 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c21ad27-9f69-41dc-bd4f-5ee4fa4bdbf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:05:33.785601 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c21ad27-9f69-41dc-bd4f-5ee4fa4bdbf7 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:05:33.794964 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 660/1346] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:05:33 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:05:38.152318 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6a523-b96f-416e-a609-b678d460a116 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:38.152855 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6a523-b96f-416e-a609-b678d460a116 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:38.152855 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c6a523-b96f-416e-a609-b678d460a116 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:38.154240 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 687/1347] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 17:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:05:38.159356 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ffd3b-7c46-4282-90e9-e49fc322d3da None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:38.159805 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ffd3b-7c46-4282-90e9-e49fc322d3da None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:38.159947 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ffd3b-7c46-4282-90e9-e49fc322d3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:38.177287 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-6e4ffd3b-7c46-4282-90e9-e49fc322d3da None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:05:38.178484 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-6e4ffd3b-7c46-4282-90e9-e49fc322d3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:05:38.186394 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 661/1348] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 17:05:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:05:38.224877 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6a523-b96f-416e-a609-b678d460a116 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:05:38.236468 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c6a523-b96f-416e-a609-b678d460a116 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:05:38.239987 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05d275-053f-4ca9-974d-c0de3d93e023 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:05:38.240639 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05d275-053f-4ca9-974d-c0de3d93e023 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:38.240925 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05d275-053f-4ca9-974d-c0de3d93e023 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:38.241198 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05d275-053f-4ca9-974d-c0de3d93e023 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:38.250695 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d05d275-053f-4ca9-974d-c0de3d93e023 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:05:38.252225 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d05d275-053f-4ca9-974d-c0de3d93e023 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:05:38.265692 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 688/1349] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:05:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:05:43.269922 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e2b094-a848-4357-83ed-a736c46e50b5 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:43.270446 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e2b094-a848-4357-83ed-a736c46e50b5 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:43.270555 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e2b094-a848-4357-83ed-a736c46e50b5 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:43.272060 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 662/1350] 10.222.0.40 () {54 vars in 921 bytes} [Wed Jul 6 17:05:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:05:43.278032 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb8a06-ba45-4c23-abcb-303a7d3c5dd4 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:43.278895 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb8a06-ba45-4c23-abcb-303a7d3c5dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:43.279350 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb8a06-ba45-4c23-abcb-303a7d3c5dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:43.371289 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-23bb8a06-ba45-4c23-abcb-303a7d3c5dd4 None None] MFA Rules not processed for user `76ca877bee9041d28ad41e45ef3dd18c`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:05:43.373610 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-23bb8a06-ba45-4c23-abcb-303a7d3c5dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:05:43.407253 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:05:43.514226 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 689/1351] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Jul 6 17:05:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:05:47.741397 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d61915-aa13-41ec-8cfb-7173c4065b60 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:47.742232 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d61915-aa13-41ec-8cfb-7173c4065b60 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:47.742754 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d61915-aa13-41ec-8cfb-7173c4065b60 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:47.744593 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 663/1352] 10.222.0.40 () {54 vars in 921 bytes} [Wed Jul 6 17:05:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:05:47.750274 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ae013-d590-4b23-8936-2d5ae7b3bda5 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:47.750738 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ae013-d590-4b23-8936-2d5ae7b3bda5 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:47.751029 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068ae013-d590-4b23-8936-2d5ae7b3bda5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:47.770253 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-068ae013-d590-4b23-8936-2d5ae7b3bda5 None None] MFA Rules not processed for user `76ca877bee9041d28ad41e45ef3dd18c`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:05:47.771789 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-068ae013-d590-4b23-8936-2d5ae7b3bda5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:05:47.782356 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 690/1353] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Jul 6 17:05:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:05:48.172598 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=72 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:05:49.267275 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7badb69-e967-4a19-b4d1-3e7e755108aa None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:49.267910 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7badb69-e967-4a19-b4d1-3e7e755108aa None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:49.267910 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7badb69-e967-4a19-b4d1-3e7e755108aa None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:49.269123 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 664/1354] 10.222.0.40 () {54 vars in 921 bytes} [Wed Jul 6 17:05:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:05:49.274095 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6d8eb-d98b-42eb-ae84-308555a50755 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:49.274548 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6d8eb-d98b-42eb-ae84-308555a50755 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:49.274801 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac6d8eb-d98b-42eb-ae84-308555a50755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:49.293577 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-7ac6d8eb-d98b-42eb-ae84-308555a50755 None None] MFA Rules not processed for user `76ca877bee9041d28ad41e45ef3dd18c`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:05:49.294840 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-7ac6d8eb-d98b-42eb-ae84-308555a50755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:05:49.304167 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 691/1355] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Jul 6 17:05:49 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:05:52.945696 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a3830-efcc-44c8-b272-ef0ca0bc3674 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:52.946212 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a3830-efcc-44c8-b272-ef0ca0bc3674 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:52.946320 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a3830-efcc-44c8-b272-ef0ca0bc3674 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:52.947619 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 665/1356] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 17:05:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:05:52.953187 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9896ac52-f80f-4fbc-aae1-7b5f1d688cb0 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:52.953633 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9896ac52-f80f-4fbc-aae1-7b5f1d688cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:52.953798 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9896ac52-f80f-4fbc-aae1-7b5f1d688cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:52.972317 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-9896ac52-f80f-4fbc-aae1-7b5f1d688cb0 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:05:52.973610 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-9896ac52-f80f-4fbc-aae1-7b5f1d688cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:05:52.981804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 692/1357] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 17:05:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:05:53.036286 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dbd249-106f-46e4-826d-f1bee4018a62 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:53.036799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dbd249-106f-46e4-826d-f1bee4018a62 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:53.037193 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dbd249-106f-46e4-826d-f1bee4018a62 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:53.038680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 666/1358] 10.222.0.40 () {54 vars in 961 bytes} [Wed Jul 6 17:05:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:05:53.047697 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ff5d9-73db-4e5a-8486-9783f2022aea None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:53.048263 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ff5d9-73db-4e5a-8486-9783f2022aea None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:53.048717 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76ff5d9-73db-4e5a-8486-9783f2022aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:53.100641 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-f76ff5d9-73db-4e5a-8486-9783f2022aea None None] MFA Rules not processed for user `af26ef96dc9d47dc91549b736939268e`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:05:53.102109 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f76ff5d9-73db-4e5a-8486-9783f2022aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:05:53.193711 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 693/1359] 10.222.0.40 () {58 vars in 1058 bytes} [Wed Jul 6 17:05:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:05:53.202675 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dbd249-106f-46e4-826d-f1bee4018a62 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:05:53.217859 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dbd249-106f-46e4-826d-f1bee4018a62 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:05:53.224018 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab86baf-2c5a-4489-8f5e-72a32eae8b83 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:05:53.224766 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab86baf-2c5a-4489-8f5e-72a32eae8b83 service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:53.225205 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab86baf-2c5a-4489-8f5e-72a32eae8b83 service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:53.225662 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab86baf-2c5a-4489-8f5e-72a32eae8b83 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:53.235060 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab86baf-2c5a-4489-8f5e-72a32eae8b83 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:05:53.236907 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab86baf-2c5a-4489-8f5e-72a32eae8b83 service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:05:53.247243 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 667/1360] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:05:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:05:55.480560 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3988a-324b-4142-9753-22f974fca533 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:55.481129 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3988a-324b-4142-9753-22f974fca533 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:55.481129 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3988a-324b-4142-9753-22f974fca533 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:55.482630 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 694/1361] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 17:05:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:05:55.487599 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795bf02e-e069-4862-ae67-e2a4dcf7358a None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:55.488036 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795bf02e-e069-4862-ae67-e2a4dcf7358a None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:55.488230 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795bf02e-e069-4862-ae67-e2a4dcf7358a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:55.506400 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-795bf02e-e069-4862-ae67-e2a4dcf7358a None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:05:55.507596 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-795bf02e-e069-4862-ae67-e2a4dcf7358a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:05:55.523800 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 668/1362] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 17:05:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:05:55.559947 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b3988a-324b-4142-9753-22f974fca533 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:05:55.574539 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b3988a-324b-4142-9753-22f974fca533 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:05:55.580703 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2092cfaa-5d53-4fdb-8356-5559768cec9d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:05:55.581339 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092cfaa-5d53-4fdb-8356-5559768cec9d service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:55.581625 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092cfaa-5d53-4fdb-8356-5559768cec9d service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:55.581897 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092cfaa-5d53-4fdb-8356-5559768cec9d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:55.590580 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2092cfaa-5d53-4fdb-8356-5559768cec9d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:05:55.592212 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2092cfaa-5d53-4fdb-8356-5559768cec9d service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:05:55.602247 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 695/1363] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:05:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:05:58.156580 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ef7e9-ef26-4331-a355-0f0c3f53353c None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:58.157115 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ef7e9-ef26-4331-a355-0f0c3f53353c None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:58.157115 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ef7e9-ef26-4331-a355-0f0c3f53353c None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:58.158502 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 669/1364] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 17:05:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:05:58.163758 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5cefd-a39d-4d7a-9276-0c7d0cea0ae9 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:58.164174 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5cefd-a39d-4d7a-9276-0c7d0cea0ae9 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:58.164517 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae5cefd-a39d-4d7a-9276-0c7d0cea0ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:58.182114 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-7ae5cefd-a39d-4d7a-9276-0c7d0cea0ae9 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:05:58.183289 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-7ae5cefd-a39d-4d7a-9276-0c7d0cea0ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:05:58.191272 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 696/1365] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 17:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:05:58.238267 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b849b-b5e4-486d-9753-7b3c83ebc4a5 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:58.238558 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b849b-b5e4-486d-9753-7b3c83ebc4a5 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:58.238844 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660b849b-b5e4-486d-9753-7b3c83ebc4a5 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:58.240247 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 670/1366] 10.222.0.40 () {54 vars in 961 bytes} [Wed Jul 6 17:05:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:05:58.245650 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038be841-da21-4c7f-ba95-10010046e9a9 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:58.248882 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038be841-da21-4c7f-ba95-10010046e9a9 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:58.248882 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038be841-da21-4c7f-ba95-10010046e9a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:58.265764 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-038be841-da21-4c7f-ba95-10010046e9a9 None None] MFA Rules not processed for user `af26ef96dc9d47dc91549b736939268e`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:05:58.267051 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-038be841-da21-4c7f-ba95-10010046e9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:05:58.276633 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 697/1367] 10.222.0.40 () {58 vars in 1058 bytes} [Wed Jul 6 17:05:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:05:58.284511 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b849b-b5e4-486d-9753-7b3c83ebc4a5 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:05:58.299142 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660b849b-b5e4-486d-9753-7b3c83ebc4a5 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:05:58.304547 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d31959-3641-4d5c-8008-f1856285234b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:05:58.305181 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d31959-3641-4d5c-8008-f1856285234b service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:05:58.305467 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d31959-3641-4d5c-8008-f1856285234b service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:05:58.305744 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d31959-3641-4d5c-8008-f1856285234b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:05:58.314633 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d31959-3641-4d5c-8008-f1856285234b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:05:58.316240 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d31959-3641-4d5c-8008-f1856285234b service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:05:58.331134 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 671/1368] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:05:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:06:00.654155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900fd9d3-f2e9-464a-8cb9-0eda3bb9b81f None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:00.654756 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900fd9d3-f2e9-464a-8cb9-0eda3bb9b81f None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:00.654756 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900fd9d3-f2e9-464a-8cb9-0eda3bb9b81f None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:00.656157 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 698/1369] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 17:06:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:06:00.662243 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04a30a-f5ba-44a4-9864-cd7594a3bd8c None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:00.662784 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04a30a-f5ba-44a4-9864-cd7594a3bd8c None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:00.663131 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a04a30a-f5ba-44a4-9864-cd7594a3bd8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:00.681335 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-2a04a30a-f5ba-44a4-9864-cd7594a3bd8c None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:06:00.682534 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-2a04a30a-f5ba-44a4-9864-cd7594a3bd8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:06:00.691098 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 672/1370] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 17:06:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:06:00.737040 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900fd9d3-f2e9-464a-8cb9-0eda3bb9b81f None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:06:00.752204 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900fd9d3-f2e9-464a-8cb9-0eda3bb9b81f None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:06:00.757910 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4358b2-a7ef-4784-b8e6-adaff66e501d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:06:00.758565 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4358b2-a7ef-4784-b8e6-adaff66e501d service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:00.758850 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4358b2-a7ef-4784-b8e6-adaff66e501d service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:00.759124 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4358b2-a7ef-4784-b8e6-adaff66e501d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:00.768785 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4358b2-a7ef-4784-b8e6-adaff66e501d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:06:00.770394 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4358b2-a7ef-4784-b8e6-adaff66e501d service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:06:00.780936 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 699/1371] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:06:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:06:01.073666 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec28328-e6e6-4395-82ad-fff55f8efffd None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:01.073957 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec28328-e6e6-4395-82ad-fff55f8efffd None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:01.074439 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec28328-e6e6-4395-82ad-fff55f8efffd None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:01.075905 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 673/1372] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 17:06:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:06:01.081836 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98100c-37c5-4f56-a9ac-ad78b00679f0 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:01.082124 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98100c-37c5-4f56-a9ac-ad78b00679f0 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:01.082574 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98100c-37c5-4f56-a9ac-ad78b00679f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:01.102120 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-2c98100c-37c5-4f56-a9ac-ad78b00679f0 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:06:01.103414 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-2c98100c-37c5-4f56-a9ac-ad78b00679f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:06:01.112055 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 700/1373] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 17:06:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:06:01.455063 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73998ad-b039-4890-b548-fc54ed90d88f None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:01.455063 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73998ad-b039-4890-b548-fc54ed90d88f None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:01.455063 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73998ad-b039-4890-b548-fc54ed90d88f None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:01.457963 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 674/1374] 10.222.0.40 () {54 vars in 963 bytes} [Wed Jul 6 17:06:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:06:01.463844 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb97948-5a94-4624-898c-ebc33db7697a None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:01.464238 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb97948-5a94-4624-898c-ebc33db7697a None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:01.464586 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb97948-5a94-4624-898c-ebc33db7697a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:01.472529 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:06:01.524625 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-feb97948-5a94-4624-898c-ebc33db7697a None None] MFA Rules not processed for user `a1bfa3e856ae46d48c82195f70aec24f`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:06:01.526270 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-feb97948-5a94-4624-898c-ebc33db7697a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:06:01.595880 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 701/1375] 10.222.0.40 () {58 vars in 1060 bytes} [Wed Jul 6 17:06:01 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:06:01.602105 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73998ad-b039-4890-b548-fc54ed90d88f None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:06:01.613805 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73998ad-b039-4890-b548-fc54ed90d88f None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:06:01.617495 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fc0eb8-ec3f-42dd-9437-73bd41c18b74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:06:01.618404 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc0eb8-ec3f-42dd-9437-73bd41c18b74 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:01.619220 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc0eb8-ec3f-42dd-9437-73bd41c18b74 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:01.619709 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fc0eb8-ec3f-42dd-9437-73bd41c18b74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:01.629900 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc0eb8-ec3f-42dd-9437-73bd41c18b74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:06:01.631708 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fc0eb8-ec3f-42dd-9437-73bd41c18b74 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:06:01.642696 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 675/1376] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:06:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:06:03.084341 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bece9b-50e6-4b9f-8c3e-be7d03de86cb None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:03.084860 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bece9b-50e6-4b9f-8c3e-be7d03de86cb None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:03.084986 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bece9b-50e6-4b9f-8c3e-be7d03de86cb None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:03.086255 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 702/1377] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 17:06:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:06:03.091345 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b9be5-dcbb-436f-839c-99aade26042d None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:03.091774 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b9be5-dcbb-436f-839c-99aade26042d None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:03.091945 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875b9be5-dcbb-436f-839c-99aade26042d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:03.109520 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-875b9be5-dcbb-436f-839c-99aade26042d None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:06:03.110704 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-875b9be5-dcbb-436f-839c-99aade26042d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:06:03.118707 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 676/1378] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 17:06:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:06:03.162671 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bece9b-50e6-4b9f-8c3e-be7d03de86cb None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:06:03.177075 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bece9b-50e6-4b9f-8c3e-be7d03de86cb None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:06:03.182446 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104e3a29-32ae-49c1-8814-abde10fc40f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:06:03.183074 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e3a29-32ae-49c1-8814-abde10fc40f8 service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:03.188066 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e3a29-32ae-49c1-8814-abde10fc40f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:03.188066 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104e3a29-32ae-49c1-8814-abde10fc40f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:03.192134 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e3a29-32ae-49c1-8814-abde10fc40f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:06:03.193688 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104e3a29-32ae-49c1-8814-abde10fc40f8 service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:06:03.202852 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 703/1379] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:06:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:06:05.495354 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa6383-7e45-4ccc-9d1e-db3622bf9ad4 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:05.495952 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa6383-7e45-4ccc-9d1e-db3622bf9ad4 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:05.495952 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa6383-7e45-4ccc-9d1e-db3622bf9ad4 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:05.497583 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 677/1380] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 17:06:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:06:05.502619 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03827ae4-0d09-45b6-ba27-ba79ef8038a7 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:05.503043 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03827ae4-0d09-45b6-ba27-ba79ef8038a7 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:05.503195 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03827ae4-0d09-45b6-ba27-ba79ef8038a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:05.522274 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-03827ae4-0d09-45b6-ba27-ba79ef8038a7 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:06:05.523671 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-03827ae4-0d09-45b6-ba27-ba79ef8038a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:06:05.533049 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 704/1381] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 17:06:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:06:05.575881 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa6383-7e45-4ccc-9d1e-db3622bf9ad4 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:06:05.591702 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa6383-7e45-4ccc-9d1e-db3622bf9ad4 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:06:05.598177 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249eca9-1f2f-40df-a6d3-c72a8f443caf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:06:05.598931 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249eca9-1f2f-40df-a6d3-c72a8f443caf service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:05.599333 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249eca9-1f2f-40df-a6d3-c72a8f443caf service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:05.599705 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249eca9-1f2f-40df-a6d3-c72a8f443caf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:05.608856 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3249eca9-1f2f-40df-a6d3-c72a8f443caf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:06:05.611153 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3249eca9-1f2f-40df-a6d3-c72a8f443caf service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:06:05.621402 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 678/1382] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:06:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:06:07.899984 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeabbed-1aa9-4585-8c5c-26ce30ad8618 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:07.900622 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeabbed-1aa9-4585-8c5c-26ce30ad8618 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:07.900622 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeabbed-1aa9-4585-8c5c-26ce30ad8618 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:07.902194 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 705/1383] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 17:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:06:07.907443 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ad29a-b5fb-4b9f-9cc9-f983b437428c None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:07.907987 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ad29a-b5fb-4b9f-9cc9-f983b437428c None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:07.908079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5ad29a-b5fb-4b9f-9cc9-f983b437428c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:07.926264 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-4b5ad29a-b5fb-4b9f-9cc9-f983b437428c None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:06:07.927407 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4b5ad29a-b5fb-4b9f-9cc9-f983b437428c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:06:07.935266 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 679/1384] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 17:06:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:06:07.976735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeabbed-1aa9-4585-8c5c-26ce30ad8618 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:06:07.992041 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aeabbed-1aa9-4585-8c5c-26ce30ad8618 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:06:07.997357 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de916b8-c6d0-40e7-9ded-89cdabf3ed25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:06:07.997995 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de916b8-c6d0-40e7-9ded-89cdabf3ed25 service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:07.998277 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de916b8-c6d0-40e7-9ded-89cdabf3ed25 service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:07.998555 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de916b8-c6d0-40e7-9ded-89cdabf3ed25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:08.007796 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de916b8-c6d0-40e7-9ded-89cdabf3ed25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:06:08.009374 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de916b8-c6d0-40e7-9ded-89cdabf3ed25 service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:06:08.019491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 706/1385] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:06:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:06:10.222528 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720f7d8-c177-4d94-b1e2-fdacea16c45a None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:10.223097 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720f7d8-c177-4d94-b1e2-fdacea16c45a None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:10.223227 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f720f7d8-c177-4d94-b1e2-fdacea16c45a None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:10.225043 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 680/1386] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 17:06:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:06:10.231456 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d061c4-1f4c-46f5-bd4d-ba73b5e80d20 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:10.231895 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d061c4-1f4c-46f5-bd4d-ba73b5e80d20 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:10.232201 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d061c4-1f4c-46f5-bd4d-ba73b5e80d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:10.253365 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-92d061c4-1f4c-46f5-bd4d-ba73b5e80d20 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:06:10.254585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-92d061c4-1f4c-46f5-bd4d-ba73b5e80d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:06:10.262584 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 707/1387] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 17:06:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:06:10.317565 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f720f7d8-c177-4d94-b1e2-fdacea16c45a None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:06:10.332639 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f720f7d8-c177-4d94-b1e2-fdacea16c45a None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:06:10.338121 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ed6a36-2430-4d88-989c-e8f697999cc0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:06:10.338763 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed6a36-2430-4d88-989c-e8f697999cc0 service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:10.339058 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed6a36-2430-4d88-989c-e8f697999cc0 service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:10.339344 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed6a36-2430-4d88-989c-e8f697999cc0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:10.347934 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ed6a36-2430-4d88-989c-e8f697999cc0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:06:10.349480 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ed6a36-2430-4d88-989c-e8f697999cc0 service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:06:10.359096 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 681/1388] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:06:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:06:12.738757 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7b726-9138-498d-a92f-1e87d8b40d37 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:12.739241 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7b726-9138-498d-a92f-1e87d8b40d37 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:12.739467 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f7b726-9138-498d-a92f-1e87d8b40d37 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:12.741245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 708/1389] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 17:06:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:06:12.747803 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7411d-6e24-43d4-96e7-ffa2f86dd756 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:12.748244 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7411d-6e24-43d4-96e7-ffa2f86dd756 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:12.748379 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b7411d-6e24-43d4-96e7-ffa2f86dd756 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:12.768970 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-23b7411d-6e24-43d4-96e7-ffa2f86dd756 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:06:12.770539 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-23b7411d-6e24-43d4-96e7-ffa2f86dd756 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:06:12.781347 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 682/1390] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 17:06:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:06:12.833832 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f7b726-9138-498d-a92f-1e87d8b40d37 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:06:12.849075 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f7b726-9138-498d-a92f-1e87d8b40d37 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:06:12.854550 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac4374a-7e0a-4720-9431-2b34ac0166f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:06:12.855198 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac4374a-7e0a-4720-9431-2b34ac0166f2 service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:12.855486 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac4374a-7e0a-4720-9431-2b34ac0166f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:12.855811 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac4374a-7e0a-4720-9431-2b34ac0166f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:12.865053 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac4374a-7e0a-4720-9431-2b34ac0166f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:06:12.866645 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac4374a-7e0a-4720-9431-2b34ac0166f2 service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:06:12.876322 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 709/1391] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:06:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:06:15.073611 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4297b8d3-3921-4c9d-88ee-6bdcf272e752 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:15.074129 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4297b8d3-3921-4c9d-88ee-6bdcf272e752 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:15.074228 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4297b8d3-3921-4c9d-88ee-6bdcf272e752 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:15.075519 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 683/1392] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 17:06:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:06:15.081215 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5cc96-5f2d-4384-bc2d-49b35b146b86 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:15.081622 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5cc96-5f2d-4384-bc2d-49b35b146b86 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:15.081777 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b5cc96-5f2d-4384-bc2d-49b35b146b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:15.099212 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-15b5cc96-5f2d-4384-bc2d-49b35b146b86 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:06:15.100427 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-15b5cc96-5f2d-4384-bc2d-49b35b146b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:06:15.108753 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 710/1393] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 17:06:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:06:15.152832 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4297b8d3-3921-4c9d-88ee-6bdcf272e752 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:06:15.167608 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4297b8d3-3921-4c9d-88ee-6bdcf272e752 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:06:15.173592 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ebf02-5eaa-44e7-959b-5c95d0dcdba1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:06:15.174256 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ebf02-5eaa-44e7-959b-5c95d0dcdba1 service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:15.174542 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ebf02-5eaa-44e7-959b-5c95d0dcdba1 service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:15.174824 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ebf02-5eaa-44e7-959b-5c95d0dcdba1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:15.184017 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15ebf02-5eaa-44e7-959b-5c95d0dcdba1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:06:15.185731 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15ebf02-5eaa-44e7-959b-5c95d0dcdba1 service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:06:15.196142 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 684/1394] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:06:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:06:17.252390 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2e164-c7c0-48e7-9a22-61efc5066267 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:17.252979 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2e164-c7c0-48e7-9a22-61efc5066267 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:17.252979 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b2e164-c7c0-48e7-9a22-61efc5066267 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:17.254327 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 711/1395] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 17:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:06:17.259513 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc83242b-f720-4d64-ab41-384452f5a43a None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:17.259967 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc83242b-f720-4d64-ab41-384452f5a43a None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:17.260203 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc83242b-f720-4d64-ab41-384452f5a43a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:17.278826 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-fc83242b-f720-4d64-ab41-384452f5a43a None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:06:17.280039 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-fc83242b-f720-4d64-ab41-384452f5a43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:06:17.287968 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 685/1396] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 17:06:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:06:17.344052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89898fb-1a59-40f6-a27f-dc02008e0dea None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:17.344349 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89898fb-1a59-40f6-a27f-dc02008e0dea None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:17.344633 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89898fb-1a59-40f6-a27f-dc02008e0dea None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:17.346000 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 712/1397] 10.222.0.40 () {54 vars in 962 bytes} [Wed Jul 6 17:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:06:17.351358 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e4f98-f1a7-44e5-ae92-4710fbf9d1e0 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:17.351655 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e4f98-f1a7-44e5-ae92-4710fbf9d1e0 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:17.351976 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e4f98-f1a7-44e5-ae92-4710fbf9d1e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:17.400967 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-929e4f98-f1a7-44e5-ae92-4710fbf9d1e0 None None] MFA Rules not processed for user `635a410f471c43109438d0c39874070b`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:06:17.402506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-929e4f98-f1a7-44e5-ae92-4710fbf9d1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:06:17.443593 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67206420-1f01-42ef-9640-d1ab4ec9d889 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:17.444319 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67206420-1f01-42ef-9640-d1ab4ec9d889 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:17.444610 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67206420-1f01-42ef-9640-d1ab4ec9d889 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:17.446155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 713/1398] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 17:06:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 17:06:17.451051 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e41eac-2c42-4ce0-9434-4e56ad4b91c0 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:17.451360 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e41eac-2c42-4ce0-9434-4e56ad4b91c0 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:17.452210 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e41eac-2c42-4ce0-9434-4e56ad4b91c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:17.463783 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 686/1399] 10.222.0.40 () {58 vars in 1059 bytes} [Wed Jul 6 17:06:17 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:06:17.470033 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-97e41eac-2c42-4ce0-9434-4e56ad4b91c0 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:06:17.471229 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-97e41eac-2c42-4ce0-9434-4e56ad4b91c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:06:17.472216 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929e4f98-f1a7-44e5-ae92-4710fbf9d1e0 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:06:17.479272 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 714/1400] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 17:06:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:06:17.479566 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929e4f98-f1a7-44e5-ae92-4710fbf9d1e0 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:06:17.483467 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d07e93e-cc70-4007-bd06-885c6aed9285 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '635a410f471c43109438d0c39874070b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:06:17.483947 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07e93e-cc70-4007-bd06-885c6aed9285 service glance] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:17.484232 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07e93e-cc70-4007-bd06-885c6aed9285 service glance] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:17.484511 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d07e93e-cc70-4007-bd06-885c6aed9285 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:17.493570 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d07e93e-cc70-4007-bd06-885c6aed9285 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:06:17.494994 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d07e93e-cc70-4007-bd06-885c6aed9285 service glance] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:06:17.504257 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 687/1401] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:06:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 17:06:17.522879 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e41eac-2c42-4ce0-9434-4e56ad4b91c0 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:06:17.536450 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e41eac-2c42-4ce0-9434-4e56ad4b91c0 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:06:17.541783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d916220e-ef86-4891-9785-a26b54886020 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:06:17.542459 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916220e-ef86-4891-9785-a26b54886020 service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:17.542742 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916220e-ef86-4891-9785-a26b54886020 service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:17.543016 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d916220e-ef86-4891-9785-a26b54886020 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:17.551278 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d916220e-ef86-4891-9785-a26b54886020 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:06:17.552888 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d916220e-ef86-4891-9785-a26b54886020 service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:06:17.562122 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 715/1402] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:06:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:06:18.367135 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec9d37-f661-4746-98ad-cb0f8470f6b7 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:18.367578 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec9d37-f661-4746-98ad-cb0f8470f6b7 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:18.367699 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dec9d37-f661-4746-98ad-cb0f8470f6b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:18.372918 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:06:18.417385 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-2dec9d37-f661-4746-98ad-cb0f8470f6b7 None None] MFA Rules not processed for user `1905e6f8c3e24e3cbfefce2827baa370`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:06:18.418622 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-2dec9d37-f661-4746-98ad-cb0f8470f6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:06:18.482204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 688/1403] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 6 17:06:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:06:18.522611 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93de162-dfb4-4792-b441-8a188ca1e893 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:18.523083 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93de162-dfb4-4792-b441-8a188ca1e893 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:18.523347 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93de162-dfb4-4792-b441-8a188ca1e893 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:18.524902 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 716/1404] 10.222.0.40 () {54 vars in 942 bytes} [Wed Jul 6 17:06:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:06:18.532521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f28be1d-d74a-441a-9593-d6a0c5ea965d None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:18.532833 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f28be1d-d74a-441a-9593-d6a0c5ea965d None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:18.533122 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f28be1d-d74a-441a-9593-d6a0c5ea965d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:18.596033 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-9f28be1d-d74a-441a-9593-d6a0c5ea965d None None] MFA Rules not processed for user `75acacd30e9a4cf2ad58b072b49e3f05`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:06:18.597618 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-9f28be1d-d74a-441a-9593-d6a0c5ea965d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:06:18.682738 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 689/1405] 10.222.0.40 () {58 vars in 1039 bytes} [Wed Jul 6 17:06:18 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:06:18.692497 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93de162-dfb4-4792-b441-8a188ca1e893 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:06:18.706172 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93de162-dfb4-4792-b441-8a188ca1e893 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:06:18.711201 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06070978-a489-49cc-88ab-d2af01f8ca40 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75acacd30e9a4cf2ad58b072b49e3f05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:06:18.711979 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06070978-a489-49cc-88ab-d2af01f8ca40 service swift] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:18.712336 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06070978-a489-49cc-88ab-d2af01f8ca40 service swift] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:18.712683 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06070978-a489-49cc-88ab-d2af01f8ca40 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:18.721827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06070978-a489-49cc-88ab-d2af01f8ca40 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:06:18.723610 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06070978-a489-49cc-88ab-d2af01f8ca40 service swift] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:06:18.732477 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 717/1406] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 6 17:06:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 17:06:19.763428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacca5d0-b4a7-49bb-9a3f-592eaec53c15 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:19.764008 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacca5d0-b4a7-49bb-9a3f-592eaec53c15 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:19.764131 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacca5d0-b4a7-49bb-9a3f-592eaec53c15 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:19.765833 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 690/1407] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 17:06:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:06:19.771357 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d1d52-a4ef-4899-ac5a-226bd52c425c None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:19.771934 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d1d52-a4ef-4899-ac5a-226bd52c425c None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:19.771934 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8d1d52-a4ef-4899-ac5a-226bd52c425c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:19.792357 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-4e8d1d52-a4ef-4899-ac5a-226bd52c425c None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:06:19.795711 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-4e8d1d52-a4ef-4899-ac5a-226bd52c425c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:06:19.805773 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 718/1408] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 17:06:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:06:19.852530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacca5d0-b4a7-49bb-9a3f-592eaec53c15 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:06:19.868244 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacca5d0-b4a7-49bb-9a3f-592eaec53c15 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:06:19.875321 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fb6d46-016a-43e3-b86a-b371195fd341 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:06:19.876092 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fb6d46-016a-43e3-b86a-b371195fd341 service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:19.876449 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fb6d46-016a-43e3-b86a-b371195fd341 service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:19.876799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fb6d46-016a-43e3-b86a-b371195fd341 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:19.885779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fb6d46-016a-43e3-b86a-b371195fd341 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:06:19.887474 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fb6d46-016a-43e3-b86a-b371195fd341 service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:06:19.897652 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 691/1409] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:06:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:06:29.787050 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=81 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:06:29.893414 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=89 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:06:42.053886 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b474c99-9f4b-4b53-8621-e7f5bbbceca7 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:42.054425 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b474c99-9f4b-4b53-8621-e7f5bbbceca7 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:42.054425 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b474c99-9f4b-4b53-8621-e7f5bbbceca7 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:42.055752 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 719/1410] 10.222.0.40 () {54 vars in 918 bytes} [Wed Jul 6 17:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:06:42.061375 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa943731-5b17-445b-b977-603681ab4369 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:42.061770 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa943731-5b17-445b-b977-603681ab4369 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:42.061932 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa943731-5b17-445b-b977-603681ab4369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:42.080040 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-fa943731-5b17-445b-b977-603681ab4369 None None] MFA Rules not processed for user `76ca877bee9041d28ad41e45ef3dd18c`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:06:42.081230 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-fa943731-5b17-445b-b977-603681ab4369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:06:42.089597 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 692/1411] 10.222.0.40 () {58 vars in 1015 bytes} [Wed Jul 6 17:06:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:06:42.103305 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61897688-afed-46d0-bc8d-3a551b6a564c None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:42.103593 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61897688-afed-46d0-bc8d-3a551b6a564c None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:42.103907 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61897688-afed-46d0-bc8d-3a551b6a564c None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:42.105314 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 720/1412] 10.222.0.40 () {54 vars in 918 bytes} [Wed Jul 6 17:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:06:42.110478 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f66235-8c17-48da-b3eb-84c3f4a13c07 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:42.110773 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f66235-8c17-48da-b3eb-84c3f4a13c07 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:42.111055 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f66235-8c17-48da-b3eb-84c3f4a13c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:42.129056 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-00f66235-8c17-48da-b3eb-84c3f4a13c07 None None] MFA Rules not processed for user `af26ef96dc9d47dc91549b736939268e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:06:42.130234 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-00f66235-8c17-48da-b3eb-84c3f4a13c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:06:42.139175 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 693/1413] 10.222.0.40 () {58 vars in 1015 bytes} [Wed Jul 6 17:06:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:06:44.614720 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ae7bd-6136-49f7-a19d-73908f7ed283 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:44.615275 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ae7bd-6136-49f7-a19d-73908f7ed283 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:44.615391 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7ae7bd-6136-49f7-a19d-73908f7ed283 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:44.616804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 721/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 17:06:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:06:44.623623 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afcf9f1-82b2-4b8c-b829-babc0b03a3fd None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:44.627184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afcf9f1-82b2-4b8c-b829-babc0b03a3fd None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:44.627184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afcf9f1-82b2-4b8c-b829-babc0b03a3fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:44.642568 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-5afcf9f1-82b2-4b8c-b829-babc0b03a3fd None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:06:44.643781 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-5afcf9f1-82b2-4b8c-b829-babc0b03a3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:06:44.652142 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 694/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 17:06:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:06:44.699420 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7ae7bd-6136-49f7-a19d-73908f7ed283 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:06:44.713493 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7ae7bd-6136-49f7-a19d-73908f7ed283 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:06:44.719366 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e8da51-98c9-44e5-ad11-52a69bd69c03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:06:44.720039 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8da51-98c9-44e5-ad11-52a69bd69c03 service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:44.720325 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8da51-98c9-44e5-ad11-52a69bd69c03 service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:44.720657 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e8da51-98c9-44e5-ad11-52a69bd69c03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:44.728922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e8da51-98c9-44e5-ad11-52a69bd69c03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:06:44.733165 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e8da51-98c9-44e5-ad11-52a69bd69c03 service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:06:44.740758 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 722/1416] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:06:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:06:45.639674 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1dc13b-b8d7-4891-96c5-94d0d0b061d1 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:45.640264 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1dc13b-b8d7-4891-96c5-94d0d0b061d1 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:45.640264 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1dc13b-b8d7-4891-96c5-94d0d0b061d1 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:45.641664 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 695/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 6 17:06:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 17:06:45.646593 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e007f94-30b9-483b-8eb0-99f56a09690c None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:45.646993 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e007f94-30b9-483b-8eb0-99f56a09690c None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:45.647151 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e007f94-30b9-483b-8eb0-99f56a09690c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:45.665019 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-8e007f94-30b9-483b-8eb0-99f56a09690c None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:06:45.666213 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-8e007f94-30b9-483b-8eb0-99f56a09690c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:06:45.673923 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 723/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 6 17:06:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:06:45.824467 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1dc13b-b8d7-4891-96c5-94d0d0b061d1 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:06:45.839352 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1dc13b-b8d7-4891-96c5-94d0d0b061d1 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:06:45.844730 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cd3b21-014c-485b-a5a6-347e2329d00a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:06:45.845365 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cd3b21-014c-485b-a5a6-347e2329d00a service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:45.845651 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cd3b21-014c-485b-a5a6-347e2329d00a service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:45.845987 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cd3b21-014c-485b-a5a6-347e2329d00a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:45.853942 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cd3b21-014c-485b-a5a6-347e2329d00a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:06:45.855550 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cd3b21-014c-485b-a5a6-347e2329d00a service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:06:45.865241 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 696/1419] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:06:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:06:46.734225 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3be9f2-11cc-4b59-9f54-fbc501b1d38a None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:46.734624 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3be9f2-11cc-4b59-9f54-fbc501b1d38a None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:46.734793 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3be9f2-11cc-4b59-9f54-fbc501b1d38a None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:46.736202 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 724/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 6 17:06:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:06:46.741404 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d821f9-e6bc-4582-b8a3-a2f627fe4d58 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:46.741816 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d821f9-e6bc-4582-b8a3-a2f627fe4d58 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:46.741973 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d821f9-e6bc-4582-b8a3-a2f627fe4d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:46.760264 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-28d821f9-e6bc-4582-b8a3-a2f627fe4d58 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:06:46.761467 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-28d821f9-e6bc-4582-b8a3-a2f627fe4d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:06:46.769477 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 697/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 6 17:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:06:46.919684 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3be9f2-11cc-4b59-9f54-fbc501b1d38a None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:06:46.938292 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3be9f2-11cc-4b59-9f54-fbc501b1d38a None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:06:46.944863 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a26ada-9925-498a-a96c-b829b5ceb87a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:06:46.945509 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a26ada-9925-498a-a96c-b829b5ceb87a service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:46.945791 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a26ada-9925-498a-a96c-b829b5ceb87a service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:46.946069 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a26ada-9925-498a-a96c-b829b5ceb87a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:46.954786 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a26ada-9925-498a-a96c-b829b5ceb87a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:06:46.956417 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a26ada-9925-498a-a96c-b829b5ceb87a service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:06:46.966038 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 725/1422] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:06:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:06:47.821697 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822aaaa0-a8b2-4c2c-9c72-033e9315c30d None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:47.822254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822aaaa0-a8b2-4c2c-9c72-033e9315c30d None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:47.822254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822aaaa0-a8b2-4c2c-9c72-033e9315c30d None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:47.823620 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 698/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 6 17:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:06:47.828990 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f8978-8de4-4ed9-8415-b34ec1a5823f None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:47.829395 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f8978-8de4-4ed9-8415-b34ec1a5823f None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:47.829544 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033f8978-8de4-4ed9-8415-b34ec1a5823f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:47.847091 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-033f8978-8de4-4ed9-8415-b34ec1a5823f None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:06:47.848299 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-033f8978-8de4-4ed9-8415-b34ec1a5823f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:06:47.855907 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 726/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 6 17:06:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:06:48.007218 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822aaaa0-a8b2-4c2c-9c72-033e9315c30d None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:06:48.021703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822aaaa0-a8b2-4c2c-9c72-033e9315c30d None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:06:48.027305 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8b312-fe9f-4b7d-91b7-94dd035a48ce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:06:48.027962 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8b312-fe9f-4b7d-91b7-94dd035a48ce service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:48.028249 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8b312-fe9f-4b7d-91b7-94dd035a48ce service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:48.028527 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8b312-fe9f-4b7d-91b7-94dd035a48ce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:48.036530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8b312-fe9f-4b7d-91b7-94dd035a48ce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:06:48.038130 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8b312-fe9f-4b7d-91b7-94dd035a48ce service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:06:48.047674 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 699/1425] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:06:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:06:48.900518 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504473f3-e5f1-43ca-ae0d-e1ea3694bf06 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:48.908706 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504473f3-e5f1-43ca-ae0d-e1ea3694bf06 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:48.908706 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504473f3-e5f1-43ca-ae0d-e1ea3694bf06 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:48.908706 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 727/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 6 17:06:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:06:48.909017 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e718f568-cf13-4eac-bf50-8ec4b1a9bb81 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:48.909017 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e718f568-cf13-4eac-bf50-8ec4b1a9bb81 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:48.909017 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e718f568-cf13-4eac-bf50-8ec4b1a9bb81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:48.926877 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-e718f568-cf13-4eac-bf50-8ec4b1a9bb81 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:06:48.928082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e718f568-cf13-4eac-bf50-8ec4b1a9bb81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:06:48.936600 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 700/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 6 17:06:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:06:49.086789 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504473f3-e5f1-43ca-ae0d-e1ea3694bf06 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:06:49.101076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504473f3-e5f1-43ca-ae0d-e1ea3694bf06 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:06:49.106346 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db221b8a-3580-46b9-9f67-8a668618eb85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:06:49.106972 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db221b8a-3580-46b9-9f67-8a668618eb85 service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:49.107826 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db221b8a-3580-46b9-9f67-8a668618eb85 service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:49.108105 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db221b8a-3580-46b9-9f67-8a668618eb85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:49.116032 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db221b8a-3580-46b9-9f67-8a668618eb85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:06:49.117623 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db221b8a-3580-46b9-9f67-8a668618eb85 service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:06:49.127396 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 728/1428] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:06:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:06:49.999317 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dbc0b1-dcce-4033-b036-a5ff15e2c7ab None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:49.999877 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dbc0b1-dcce-4033-b036-a5ff15e2c7ab None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:49.999877 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dbc0b1-dcce-4033-b036-a5ff15e2c7ab None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:50.001177 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 701/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 6 17:06:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:06:50.006021 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593a6bc-a35d-4fcd-8f87-5a6846af1d38 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:50.006440 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593a6bc-a35d-4fcd-8f87-5a6846af1d38 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:50.006589 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1593a6bc-a35d-4fcd-8f87-5a6846af1d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:50.025108 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-1593a6bc-a35d-4fcd-8f87-5a6846af1d38 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:06:50.026579 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-1593a6bc-a35d-4fcd-8f87-5a6846af1d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:06:50.034432 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 729/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 6 17:06:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:06:50.184194 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dbc0b1-dcce-4033-b036-a5ff15e2c7ab None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:06:50.198745 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dbc0b1-dcce-4033-b036-a5ff15e2c7ab None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:06:50.204673 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466ebb1b-e478-4c4f-ac9e-dffb9f940a21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:06:50.205299 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466ebb1b-e478-4c4f-ac9e-dffb9f940a21 service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:50.205583 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466ebb1b-e478-4c4f-ac9e-dffb9f940a21 service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:50.205861 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466ebb1b-e478-4c4f-ac9e-dffb9f940a21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:50.213828 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466ebb1b-e478-4c4f-ac9e-dffb9f940a21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:06:50.215403 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466ebb1b-e478-4c4f-ac9e-dffb9f940a21 service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:06:50.224554 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 702/1431] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:06:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:06:51.096265 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bbb48-b029-4e80-b2ce-26e74a24c8fd None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:51.096747 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bbb48-b029-4e80-b2ce-26e74a24c8fd None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:51.096859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213bbb48-b029-4e80-b2ce-26e74a24c8fd None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:51.098200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 730/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 6 17:06:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:06:51.103138 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188b497-92e6-413d-add5-8ad1d1ab87ac None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:51.103579 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188b497-92e6-413d-add5-8ad1d1ab87ac None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:51.103703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188b497-92e6-413d-add5-8ad1d1ab87ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:51.121609 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-e188b497-92e6-413d-add5-8ad1d1ab87ac None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:06:51.122795 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e188b497-92e6-413d-add5-8ad1d1ab87ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:06:51.130579 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 703/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 6 17:06:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:06:51.287028 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213bbb48-b029-4e80-b2ce-26e74a24c8fd None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:06:51.301144 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213bbb48-b029-4e80-b2ce-26e74a24c8fd None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:06:51.306468 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75579a89-834f-4b70-b9ef-3c1305d78f12 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:06:51.307100 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75579a89-834f-4b70-b9ef-3c1305d78f12 service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:51.307382 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75579a89-834f-4b70-b9ef-3c1305d78f12 service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:51.307659 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75579a89-834f-4b70-b9ef-3c1305d78f12 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:51.315612 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75579a89-834f-4b70-b9ef-3c1305d78f12 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:06:51.317223 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75579a89-834f-4b70-b9ef-3c1305d78f12 service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:06:51.327152 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 731/1434] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:06:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:06:52.161554 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209c3e3-c118-498d-b1cc-fe45cfef1bbc None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:52.162059 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209c3e3-c118-498d-b1cc-fe45cfef1bbc None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:52.162168 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c209c3e3-c118-498d-b1cc-fe45cfef1bbc None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:52.163458 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 704/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 6 17:06:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:06:52.168378 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a448e93-ec1c-43b0-9de7-3f15a2a8916a None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:52.168830 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a448e93-ec1c-43b0-9de7-3f15a2a8916a None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:52.168929 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a448e93-ec1c-43b0-9de7-3f15a2a8916a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:52.187618 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-4a448e93-ec1c-43b0-9de7-3f15a2a8916a None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:06:52.188832 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-4a448e93-ec1c-43b0-9de7-3f15a2a8916a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:06:52.196527 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 732/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 6 17:06:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:06:52.344089 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209c3e3-c118-498d-b1cc-fe45cfef1bbc None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:06:52.358570 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c209c3e3-c118-498d-b1cc-fe45cfef1bbc None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:06:52.363864 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be253e6-5398-4520-8304-b7e9a1817307 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:06:52.364434 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be253e6-5398-4520-8304-b7e9a1817307 service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:52.364716 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be253e6-5398-4520-8304-b7e9a1817307 service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:52.364990 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be253e6-5398-4520-8304-b7e9a1817307 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:52.372858 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be253e6-5398-4520-8304-b7e9a1817307 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:06:52.374429 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be253e6-5398-4520-8304-b7e9a1817307 service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:06:52.383506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 705/1437] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:06:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:06:53.266162 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a31fda-6c95-42a5-baf2-7de5f3a0afe5 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:53.266730 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a31fda-6c95-42a5-baf2-7de5f3a0afe5 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:53.266730 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a31fda-6c95-42a5-baf2-7de5f3a0afe5 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:53.268118 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 733/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 6 17:06:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:06:53.273069 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e357b-af68-4a79-adce-a6d003cf4e9b None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:53.273489 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e357b-af68-4a79-adce-a6d003cf4e9b None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:53.273635 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698e357b-af68-4a79-adce-a6d003cf4e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:53.291476 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-698e357b-af68-4a79-adce-a6d003cf4e9b None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:06:53.292632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-698e357b-af68-4a79-adce-a6d003cf4e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:06:53.300346 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 706/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 6 17:06:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:06:53.450498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a31fda-6c95-42a5-baf2-7de5f3a0afe5 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:06:53.464818 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a31fda-6c95-42a5-baf2-7de5f3a0afe5 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:06:53.470229 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6e87a3-8c73-4b1d-9a4f-14446f921ab8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:06:53.470860 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e87a3-8c73-4b1d-9a4f-14446f921ab8 service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:53.471797 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e87a3-8c73-4b1d-9a4f-14446f921ab8 service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:53.471797 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6e87a3-8c73-4b1d-9a4f-14446f921ab8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:53.479417 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6e87a3-8c73-4b1d-9a4f-14446f921ab8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:06:53.481145 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6e87a3-8c73-4b1d-9a4f-14446f921ab8 service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:06:53.491033 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 734/1440] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:06:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:06:55.426133 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bfc640-625b-482f-9385-305db3d4288f None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:55.426736 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bfc640-625b-482f-9385-305db3d4288f None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:55.426736 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bfc640-625b-482f-9385-305db3d4288f None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:55.428152 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 707/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 17:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:06:55.433306 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846fae3-714b-4e68-811c-fc0f8137bc9b None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:55.433784 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846fae3-714b-4e68-811c-fc0f8137bc9b None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:55.433892 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1846fae3-714b-4e68-811c-fc0f8137bc9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:55.453566 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-1846fae3-714b-4e68-811c-fc0f8137bc9b None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:06:55.454770 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-1846fae3-714b-4e68-811c-fc0f8137bc9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:06:55.462629 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 735/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 17:06:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:06:55.500469 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bfc640-625b-482f-9385-305db3d4288f None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:06:55.511201 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bfc640-625b-482f-9385-305db3d4288f None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:06:55.515371 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c54a18-8a13-42a6-9b2d-74f2027b7832 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:06:55.516044 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c54a18-8a13-42a6-9b2d-74f2027b7832 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:55.516332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c54a18-8a13-42a6-9b2d-74f2027b7832 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:55.517015 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c54a18-8a13-42a6-9b2d-74f2027b7832 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:55.525413 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c54a18-8a13-42a6-9b2d-74f2027b7832 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:06:55.527126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c54a18-8a13-42a6-9b2d-74f2027b7832 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:06:55.536467 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 708/1443] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:06:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:06:57.681752 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab67e2-05c7-4196-8204-bc5c3c5a41ce None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:57.682305 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab67e2-05c7-4196-8204-bc5c3c5a41ce None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:57.682425 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ab67e2-05c7-4196-8204-bc5c3c5a41ce None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:57.683796 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 736/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 17:06:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:06:57.689139 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b26a3d-3d66-41c7-b83e-2450ed97eb8d None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:57.689540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b26a3d-3d66-41c7-b83e-2450ed97eb8d None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:57.689707 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b26a3d-3d66-41c7-b83e-2450ed97eb8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:57.707020 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-50b26a3d-3d66-41c7-b83e-2450ed97eb8d None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:06:57.708231 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-50b26a3d-3d66-41c7-b83e-2450ed97eb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:06:57.719742 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 709/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 17:06:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:06:57.754430 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ab67e2-05c7-4196-8204-bc5c3c5a41ce None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:06:57.765001 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ab67e2-05c7-4196-8204-bc5c3c5a41ce None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:06:57.768644 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735f9eba-35b6-4c3e-ae80-e7e87b12d83a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:06:57.769317 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f9eba-35b6-4c3e-ae80-e7e87b12d83a service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:06:57.769604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f9eba-35b6-4c3e-ae80-e7e87b12d83a service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:06:57.769927 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735f9eba-35b6-4c3e-ae80-e7e87b12d83a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:06:57.779700 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735f9eba-35b6-4c3e-ae80-e7e87b12d83a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:06:57.781179 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735f9eba-35b6-4c3e-ae80-e7e87b12d83a service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:06:57.790635 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 737/1446] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:06:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:07:00.157753 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b5d54-770a-4454-8aab-488aec82b5f7 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:00.158336 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b5d54-770a-4454-8aab-488aec82b5f7 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:00.158336 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b5d54-770a-4454-8aab-488aec82b5f7 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:00.159742 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 710/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 17:07:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:07:00.164940 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138e292-d18b-4014-a004-8e9ce7010ef4 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:00.165367 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138e292-d18b-4014-a004-8e9ce7010ef4 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:00.165503 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f138e292-d18b-4014-a004-8e9ce7010ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:00.297269 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-f138e292-d18b-4014-a004-8e9ce7010ef4 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:07:00.298495 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f138e292-d18b-4014-a004-8e9ce7010ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:07:00.306180 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 738/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 17:07:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:07:00.344336 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4b5d54-770a-4454-8aab-488aec82b5f7 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:07:00.354953 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4b5d54-770a-4454-8aab-488aec82b5f7 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:07:00.359233 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d87587-ff61-43e6-ab71-0f5165655130 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:07:00.359922 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d87587-ff61-43e6-ab71-0f5165655130 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:00.360207 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d87587-ff61-43e6-ab71-0f5165655130 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:00.360486 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d87587-ff61-43e6-ab71-0f5165655130 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:00.369448 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d87587-ff61-43e6-ab71-0f5165655130 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:07:00.370863 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d87587-ff61-43e6-ab71-0f5165655130 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:07:00.380731 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 711/1449] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:07:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:07:02.524229 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def8cd5-585a-4923-80c9-1806e71208d0 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:02.524799 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def8cd5-585a-4923-80c9-1806e71208d0 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:02.524799 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9def8cd5-585a-4923-80c9-1806e71208d0 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:02.526402 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 739/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 17:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:07:02.531794 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb6163-5957-4e2e-9f4b-b1cd5e6d262f None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:02.532215 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb6163-5957-4e2e-9f4b-b1cd5e6d262f None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:02.532367 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeb6163-5957-4e2e-9f4b-b1cd5e6d262f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:02.550201 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-ddeb6163-5957-4e2e-9f4b-b1cd5e6d262f None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:07:02.551390 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ddeb6163-5957-4e2e-9f4b-b1cd5e6d262f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:07:02.559168 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 712/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 17:07:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:07:02.597270 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9def8cd5-585a-4923-80c9-1806e71208d0 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:07:02.608170 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9def8cd5-585a-4923-80c9-1806e71208d0 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:07:02.611590 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007e74d-0c10-4147-9939-823c85497489 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:07:02.612258 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007e74d-0c10-4147-9939-823c85497489 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:02.612541 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007e74d-0c10-4147-9939-823c85497489 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:02.612817 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007e74d-0c10-4147-9939-823c85497489 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:02.621436 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c007e74d-0c10-4147-9939-823c85497489 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:07:02.623461 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c007e74d-0c10-4147-9939-823c85497489 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:07:02.632657 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 740/1452] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:07:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:07:04.701607 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405971ca-8fa1-40c0-888e-bdb315534532 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:04.702157 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405971ca-8fa1-40c0-888e-bdb315534532 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:04.702157 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405971ca-8fa1-40c0-888e-bdb315534532 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:04.703534 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 713/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 17:07:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:07:04.708815 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c0980-60d1-40c6-8751-c5d596f63692 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:04.709235 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c0980-60d1-40c6-8751-c5d596f63692 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:04.709426 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c0980-60d1-40c6-8751-c5d596f63692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:04.726800 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-1a0c0980-60d1-40c6-8751-c5d596f63692 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:07:04.728031 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-1a0c0980-60d1-40c6-8751-c5d596f63692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:07:04.739612 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 741/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 17:07:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:07:04.777583 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405971ca-8fa1-40c0-888e-bdb315534532 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:07:04.788449 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405971ca-8fa1-40c0-888e-bdb315534532 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:07:04.791927 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb03ca2b-2d62-4ddd-9663-1ad13cd368fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:07:04.792557 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03ca2b-2d62-4ddd-9663-1ad13cd368fc service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:04.792842 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03ca2b-2d62-4ddd-9663-1ad13cd368fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:04.793116 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03ca2b-2d62-4ddd-9663-1ad13cd368fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:04.802614 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb03ca2b-2d62-4ddd-9663-1ad13cd368fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:07:04.804077 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb03ca2b-2d62-4ddd-9663-1ad13cd368fc service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:07:04.814292 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 714/1455] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:07:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:07:06.881880 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3890be1-cb92-421b-839f-5f22a8631e70 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:06.882437 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3890be1-cb92-421b-839f-5f22a8631e70 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:06.882437 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3890be1-cb92-421b-839f-5f22a8631e70 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:06.883806 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 742/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 17:07:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:07:06.889471 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2e269-fe2a-4f2f-96ae-c69498dab201 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:06.889924 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2e269-fe2a-4f2f-96ae-c69498dab201 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:06.890031 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2e269-fe2a-4f2f-96ae-c69498dab201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:06.907963 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-cfe2e269-fe2a-4f2f-96ae-c69498dab201 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:07:06.909140 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-cfe2e269-fe2a-4f2f-96ae-c69498dab201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:07:06.916693 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 715/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 17:07:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:07:06.953955 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3890be1-cb92-421b-839f-5f22a8631e70 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:07:06.964355 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3890be1-cb92-421b-839f-5f22a8631e70 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:07:06.968537 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01928d9-9596-4231-9266-9a4af8fdefc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:07:06.969174 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01928d9-9596-4231-9266-9a4af8fdefc7 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:06.969457 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01928d9-9596-4231-9266-9a4af8fdefc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:06.969730 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01928d9-9596-4231-9266-9a4af8fdefc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:06.978439 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01928d9-9596-4231-9266-9a4af8fdefc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:07:06.979883 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01928d9-9596-4231-9266-9a4af8fdefc7 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:07:06.989219 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 743/1458] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:07:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:07:09.043809 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09f00b-6603-49fa-9f4b-1291fce74bbf None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:09.044267 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09f00b-6603-49fa-9f4b-1291fce74bbf None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:09.044374 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf09f00b-6603-49fa-9f4b-1291fce74bbf None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:09.045835 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 716/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 17:07:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:07:09.051006 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d50e23-a119-4747-8ec6-b1f5f0aca6aa None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:09.051428 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d50e23-a119-4747-8ec6-b1f5f0aca6aa None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:09.051563 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d50e23-a119-4747-8ec6-b1f5f0aca6aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:09.069769 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-20d50e23-a119-4747-8ec6-b1f5f0aca6aa None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:07:09.071028 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-20d50e23-a119-4747-8ec6-b1f5f0aca6aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:07:09.079490 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 744/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 17:07:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:07:09.116502 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf09f00b-6603-49fa-9f4b-1291fce74bbf None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:07:09.127380 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf09f00b-6603-49fa-9f4b-1291fce74bbf None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:07:09.130807 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dec910c-1dd6-48dd-9f9a-08537b0cff7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:07:09.131435 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec910c-1dd6-48dd-9f9a-08537b0cff7f service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:09.131716 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec910c-1dd6-48dd-9f9a-08537b0cff7f service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:09.132022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dec910c-1dd6-48dd-9f9a-08537b0cff7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:09.140579 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dec910c-1dd6-48dd-9f9a-08537b0cff7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:07:09.141990 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dec910c-1dd6-48dd-9f9a-08537b0cff7f service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:07:09.151106 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 717/1461] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:07:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:07:11.203268 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2770f2-07d7-4e78-98df-57badbf28a19 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:11.203852 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2770f2-07d7-4e78-98df-57badbf28a19 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:11.203852 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2770f2-07d7-4e78-98df-57badbf28a19 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:11.205268 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 745/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 17:07:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:07:11.210517 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a939e9-60db-4895-9f66-d6cf29f2f7ff None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:11.210948 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a939e9-60db-4895-9f66-d6cf29f2f7ff None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:11.211092 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a939e9-60db-4895-9f66-d6cf29f2f7ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:11.217651 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=79 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:07:11.229322 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-27a939e9-60db-4895-9f66-d6cf29f2f7ff None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:07:11.230662 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-27a939e9-60db-4895-9f66-d6cf29f2f7ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:07:11.239116 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 718/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 17:07:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:07:11.275963 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2770f2-07d7-4e78-98df-57badbf28a19 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:07:11.277963 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-db2770f2-07d7-4e78-98df-57badbf28a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:07:11.286196 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=73 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:07:11.292091 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2770f2-07d7-4e78-98df-57badbf28a19 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:07:11.296786 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed05b092-0ed3-419d-93de-2d1ed9c854be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:07:11.296933 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05b092-0ed3-419d-93de-2d1ed9c854be service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:11.297209 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05b092-0ed3-419d-93de-2d1ed9c854be service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:11.297483 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed05b092-0ed3-419d-93de-2d1ed9c854be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:11.306237 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed05b092-0ed3-419d-93de-2d1ed9c854be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:07:11.307655 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed05b092-0ed3-419d-93de-2d1ed9c854be service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:07:11.317216 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 746/1464] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:07:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:07:13.383343 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d8b01-206c-47dc-b4ea-3654cb94e3d3 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:13.383951 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d8b01-206c-47dc-b4ea-3654cb94e3d3 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:13.383951 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4d8b01-206c-47dc-b4ea-3654cb94e3d3 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:13.385394 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 719/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 17:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:07:13.392091 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353d0c7-484f-4ec5-bfe3-999365246ca7 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:13.392091 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353d0c7-484f-4ec5-bfe3-999365246ca7 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:13.392091 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c353d0c7-484f-4ec5-bfe3-999365246ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:13.409721 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-c353d0c7-484f-4ec5-bfe3-999365246ca7 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:07:13.410967 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c353d0c7-484f-4ec5-bfe3-999365246ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:07:13.419512 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 747/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 17:07:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:07:13.455772 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4d8b01-206c-47dc-b4ea-3654cb94e3d3 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:07:13.467217 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4d8b01-206c-47dc-b4ea-3654cb94e3d3 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:07:13.470754 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1808abe4-eaad-47e2-b464-b6ee9f8f3f38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:07:13.471394 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1808abe4-eaad-47e2-b464-b6ee9f8f3f38 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:13.471681 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1808abe4-eaad-47e2-b464-b6ee9f8f3f38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:13.472001 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1808abe4-eaad-47e2-b464-b6ee9f8f3f38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:13.480998 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1808abe4-eaad-47e2-b464-b6ee9f8f3f38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:07:13.483013 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1808abe4-eaad-47e2-b464-b6ee9f8f3f38 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:07:13.492398 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 720/1467] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:07:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:07:15.521607 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317927c-2213-4561-abbe-5ae4819ae08d None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:15.522198 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317927c-2213-4561-abbe-5ae4819ae08d None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:15.522198 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317927c-2213-4561-abbe-5ae4819ae08d None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:15.523528 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 748/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 17:07:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:07:15.529006 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f134dde-db8e-4d85-b33c-f282a54df5cf None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:15.529391 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f134dde-db8e-4d85-b33c-f282a54df5cf None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:15.529576 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f134dde-db8e-4d85-b33c-f282a54df5cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:15.547141 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-8f134dde-db8e-4d85-b33c-f282a54df5cf None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:07:15.548366 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-8f134dde-db8e-4d85-b33c-f282a54df5cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:07:15.556089 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 721/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 17:07:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:07:15.592281 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e317927c-2213-4561-abbe-5ae4819ae08d None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:07:15.602890 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e317927c-2213-4561-abbe-5ae4819ae08d None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:07:15.606426 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd045a1f-7edd-4f6c-a925-b5dc6ee6d5ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:07:15.607056 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd045a1f-7edd-4f6c-a925-b5dc6ee6d5ed service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:15.607343 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd045a1f-7edd-4f6c-a925-b5dc6ee6d5ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:15.607618 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd045a1f-7edd-4f6c-a925-b5dc6ee6d5ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:15.616448 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd045a1f-7edd-4f6c-a925-b5dc6ee6d5ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:07:15.617861 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd045a1f-7edd-4f6c-a925-b5dc6ee6d5ed service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:07:15.628066 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 749/1470] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:07:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:07:25.542139 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=11 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:07:25.623807 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=14 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:07:35.637409 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743396e6-e626-4085-ab47-0d50c0378a07 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:35.637988 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743396e6-e626-4085-ab47-0d50c0378a07 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:35.637988 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743396e6-e626-4085-ab47-0d50c0378a07 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:35.639386 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 722/1471] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 17:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:07:35.644611 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1136ec0d-e7d9-4dfe-ba91-cb55ed2ee4ba None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:35.645034 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1136ec0d-e7d9-4dfe-ba91-cb55ed2ee4ba None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:35.645146 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1136ec0d-e7d9-4dfe-ba91-cb55ed2ee4ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:35.663406 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-1136ec0d-e7d9-4dfe-ba91-cb55ed2ee4ba None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:07:35.664626 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-1136ec0d-e7d9-4dfe-ba91-cb55ed2ee4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:07:35.673058 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 750/1472] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 17:07:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:07:35.756455 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1fdaf-112e-4bc7-8d87-90a02f0cf677 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:35.756749 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1fdaf-112e-4bc7-8d87-90a02f0cf677 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:35.757026 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1fdaf-112e-4bc7-8d87-90a02f0cf677 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:35.758392 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 723/1473] 10.222.0.40 () {54 vars in 962 bytes} [Wed Jul 6 17:07:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:07:35.763330 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277da194-db5f-4dff-8b1d-c9019c889525 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:35.763614 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277da194-db5f-4dff-8b1d-c9019c889525 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:35.763919 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277da194-db5f-4dff-8b1d-c9019c889525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:35.782352 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-277da194-db5f-4dff-8b1d-c9019c889525 None None] MFA Rules not processed for user `635a410f471c43109438d0c39874070b`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:07:35.783532 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-277da194-db5f-4dff-8b1d-c9019c889525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:07:35.789858 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 751/1474] 10.222.0.40 () {58 vars in 1059 bytes} [Wed Jul 6 17:07:35 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:07:35.797572 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a1fdaf-112e-4bc7-8d87-90a02f0cf677 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:07:35.808039 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a1fdaf-112e-4bc7-8d87-90a02f0cf677 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:07:35.811512 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6570be1-3ea1-4276-a988-3cf58a9fac00 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '635a410f471c43109438d0c39874070b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:07:35.812208 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6570be1-3ea1-4276-a988-3cf58a9fac00 service glance] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:35.812491 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6570be1-3ea1-4276-a988-3cf58a9fac00 service glance] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:35.812769 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6570be1-3ea1-4276-a988-3cf58a9fac00 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:35.821434 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6570be1-3ea1-4276-a988-3cf58a9fac00 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:07:35.822855 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6570be1-3ea1-4276-a988-3cf58a9fac00 service glance] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:07:35.832197 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 724/1475] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:07:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:07:37.939190 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f384d-9bb7-4fd9-82d3-8ae8bed93e49 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:37.939761 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f384d-9bb7-4fd9-82d3-8ae8bed93e49 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:37.939761 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99f384d-9bb7-4fd9-82d3-8ae8bed93e49 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:37.941129 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 752/1476] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 17:07:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:07:37.946237 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3d09b-894f-4e8e-b80d-3099dceb0673 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:37.946643 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3d09b-894f-4e8e-b80d-3099dceb0673 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:37.946927 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de3d09b-894f-4e8e-b80d-3099dceb0673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:37.964254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-8de3d09b-894f-4e8e-b80d-3099dceb0673 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:07:37.965538 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-8de3d09b-894f-4e8e-b80d-3099dceb0673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:07:37.973635 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 725/1477] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 17:07:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:07:37.996917 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f384d-9bb7-4fd9-82d3-8ae8bed93e49 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:07:38.009147 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99f384d-9bb7-4fd9-82d3-8ae8bed93e49 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:07:38.013596 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd3af59-ba85-4afd-8036-830331c6769b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:07:38.014237 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3af59-ba85-4afd-8036-830331c6769b admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:38.014520 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3af59-ba85-4afd-8036-830331c6769b admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:38.014796 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3af59-ba85-4afd-8036-830331c6769b admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:38.015527 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd3af59-ba85-4afd-8036-830331c6769b admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:07:38.017280 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: WARNING py.warnings [None req-6fd3af59-ba85-4afd-8036-830331c6769b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 06 17:07:38.017280 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: warnings.warn(msg) Jul 06 17:07:38.017280 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: Jul 06 17:07:38.018416 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd3af59-ba85-4afd-8036-830331c6769b admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:07:38.037691 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 753/1478] 10.222.0.40 () {56 vars in 1108 bytes} [Wed Jul 6 17:07:37 2022] GET /identity/v3/projects => generated 2553 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:07:40.093986 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d117a5-b1b2-4cde-a1be-c582f2a669b3 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:40.094499 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d117a5-b1b2-4cde-a1be-c582f2a669b3 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:40.094499 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d117a5-b1b2-4cde-a1be-c582f2a669b3 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:40.095848 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 726/1479] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 17:07:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:07:40.101253 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d5c87-0602-4dd4-9ad6-2016ba94fff8 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:40.101693 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d5c87-0602-4dd4-9ad6-2016ba94fff8 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:40.101824 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d5c87-0602-4dd4-9ad6-2016ba94fff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:40.119178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-903d5c87-0602-4dd4-9ad6-2016ba94fff8 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:07:40.120509 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-903d5c87-0602-4dd4-9ad6-2016ba94fff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:07:40.128076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 754/1480] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 17:07:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:07:40.173082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d117a5-b1b2-4cde-a1be-c582f2a669b3 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:07:40.188645 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d117a5-b1b2-4cde-a1be-c582f2a669b3 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:07:40.194231 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5110fd6-18af-4be6-ae44-2f12276f1b34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:07:40.194908 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5110fd6-18af-4be6-ae44-2f12276f1b34 service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:40.195235 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5110fd6-18af-4be6-ae44-2f12276f1b34 service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:40.195506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5110fd6-18af-4be6-ae44-2f12276f1b34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:40.204841 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5110fd6-18af-4be6-ae44-2f12276f1b34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:07:40.206410 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5110fd6-18af-4be6-ae44-2f12276f1b34 service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:07:40.215752 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 727/1481] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:07:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:07:42.395510 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130b6be-6969-4203-8a43-2c97d5275ca0 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:42.396081 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130b6be-6969-4203-8a43-2c97d5275ca0 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:42.396081 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2130b6be-6969-4203-8a43-2c97d5275ca0 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:42.397378 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 755/1482] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 17:07:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:07:42.402899 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b3377-e540-4c77-aa16-39268e51e9c5 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:42.403374 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b3377-e540-4c77-aa16-39268e51e9c5 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:42.403523 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b3377-e540-4c77-aa16-39268e51e9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:42.420829 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-957b3377-e540-4c77-aa16-39268e51e9c5 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:07:42.422003 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-957b3377-e540-4c77-aa16-39268e51e9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:07:42.429577 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 728/1483] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 17:07:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:07:42.465376 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2130b6be-6969-4203-8a43-2c97d5275ca0 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:07:42.476064 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2130b6be-6969-4203-8a43-2c97d5275ca0 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:07:42.479603 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae475ee-7302-4eaa-81e1-85603d70e328 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:07:42.480274 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae475ee-7302-4eaa-81e1-85603d70e328 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:42.480559 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae475ee-7302-4eaa-81e1-85603d70e328 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:42.480836 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae475ee-7302-4eaa-81e1-85603d70e328 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:42.489637 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae475ee-7302-4eaa-81e1-85603d70e328 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:07:42.491053 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae475ee-7302-4eaa-81e1-85603d70e328 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:07:42.500726 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 756/1484] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:07:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:07:44.577139 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15803a6b-0b87-4db7-9bda-a13d1086e071 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:44.577711 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15803a6b-0b87-4db7-9bda-a13d1086e071 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:44.577711 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15803a6b-0b87-4db7-9bda-a13d1086e071 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:44.579079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 729/1485] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 17:07:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:07:44.584326 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c6b93-cd12-4184-985c-01e5918bd9fb None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:44.584755 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c6b93-cd12-4184-985c-01e5918bd9fb None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:44.584893 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c6b93-cd12-4184-985c-01e5918bd9fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:44.602835 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-345c6b93-cd12-4184-985c-01e5918bd9fb None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:07:44.604062 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-345c6b93-cd12-4184-985c-01e5918bd9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:07:44.611839 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 757/1486] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 17:07:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:07:44.649455 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15803a6b-0b87-4db7-9bda-a13d1086e071 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:07:44.660879 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15803a6b-0b87-4db7-9bda-a13d1086e071 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:07:44.664428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bff112-070e-4eb4-b678-84633f76db77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:07:44.665067 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bff112-070e-4eb4-b678-84633f76db77 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:44.665355 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bff112-070e-4eb4-b678-84633f76db77 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:44.665632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bff112-070e-4eb4-b678-84633f76db77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:44.676483 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bff112-070e-4eb4-b678-84633f76db77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:07:44.678235 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bff112-070e-4eb4-b678-84633f76db77 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:07:44.688642 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 730/1487] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:07:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:07:46.805277 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a34aed-b4d4-4826-b06c-61d236f155da None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:46.805796 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a34aed-b4d4-4826-b06c-61d236f155da None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:46.805894 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a34aed-b4d4-4826-b06c-61d236f155da None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:46.807205 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 758/1488] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 17:07:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:07:46.812478 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a19c3-a4e5-4317-9241-86ee9a8311ba None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:46.812889 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a19c3-a4e5-4317-9241-86ee9a8311ba None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:46.813035 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7a19c3-a4e5-4317-9241-86ee9a8311ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:46.830611 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-cc7a19c3-a4e5-4317-9241-86ee9a8311ba None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:07:46.831827 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-cc7a19c3-a4e5-4317-9241-86ee9a8311ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:07:46.839447 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 731/1489] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 17:07:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:07:46.878817 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a34aed-b4d4-4826-b06c-61d236f155da None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:07:46.890052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a34aed-b4d4-4826-b06c-61d236f155da None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:07:46.893541 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c0f3a2-dfbd-4b45-ba9f-98893ac80461 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:07:46.899883 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c0f3a2-dfbd-4b45-ba9f-98893ac80461 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:46.899883 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c0f3a2-dfbd-4b45-ba9f-98893ac80461 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:46.899883 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c0f3a2-dfbd-4b45-ba9f-98893ac80461 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:46.904326 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c0f3a2-dfbd-4b45-ba9f-98893ac80461 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:07:46.905749 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c0f3a2-dfbd-4b45-ba9f-98893ac80461 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:07:46.915080 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 759/1490] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:07:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:07:49.339942 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b23cb9-5dce-4a0a-b379-a4d523cef14c None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:49.341013 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b23cb9-5dce-4a0a-b379-a4d523cef14c None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:49.341013 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b23cb9-5dce-4a0a-b379-a4d523cef14c None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:49.342290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 732/1491] 10.222.0.40 () {54 vars in 926 bytes} [Wed Jul 6 17:07:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:07:49.347490 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b424dc-2f09-40c2-bb07-26b2d5878858 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:49.347921 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b424dc-2f09-40c2-bb07-26b2d5878858 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:49.348112 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b424dc-2f09-40c2-bb07-26b2d5878858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:49.366375 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-72b424dc-2f09-40c2-bb07-26b2d5878858 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:07:49.367561 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-72b424dc-2f09-40c2-bb07-26b2d5878858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:07:49.375358 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 760/1492] 10.222.0.40 () {58 vars in 1023 bytes} [Wed Jul 6 17:07:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:07:49.413918 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b23cb9-5dce-4a0a-b379-a4d523cef14c None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:07:49.425106 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b23cb9-5dce-4a0a-b379-a4d523cef14c None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:07:49.428750 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e22dec4-8d53-4a56-a433-56f053a7701d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:07:49.429386 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22dec4-8d53-4a56-a433-56f053a7701d service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:07:49.429671 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22dec4-8d53-4a56-a433-56f053a7701d service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:07:49.429948 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22dec4-8d53-4a56-a433-56f053a7701d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:07:49.438737 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e22dec4-8d53-4a56-a433-56f053a7701d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:07:49.440222 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e22dec4-8d53-4a56-a433-56f053a7701d service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:07:49.449439 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 733/1493] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:07:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:07:59.361912 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=33 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:07:59.445336 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=33 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:09:30.839390 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8985b2-8e87-4652-b7f0-7cdde3acda62 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:09:30.839980 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8985b2-8e87-4652-b7f0-7cdde3acda62 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:09:30.839980 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8985b2-8e87-4652-b7f0-7cdde3acda62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:09:30.857408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-ad8985b2-8e87-4652-b7f0-7cdde3acda62 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:09:30.858584 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-ad8985b2-8e87-4652-b7f0-7cdde3acda62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:09:30.866168 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 761/1494] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:09:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:09:30.874458 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ad07c-43ae-4737-82db-94d3e9f913b6 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:09:30.874937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ad07c-43ae-4737-82db-94d3e9f913b6 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:09:30.875042 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96ad07c-43ae-4737-82db-94d3e9f913b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:09:30.894343 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-b96ad07c-43ae-4737-82db-94d3e9f913b6 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:09:30.895531 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-b96ad07c-43ae-4737-82db-94d3e9f913b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:09:30.903622 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 734/1495] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:09:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:09:30.907914 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8985b2-8e87-4652-b7f0-7cdde3acda62 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:09:30.917246 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8985b2-8e87-4652-b7f0-7cdde3acda62 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:09:30.921617 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00065f28-3393-4e02-8a8f-95ab91c100f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:09:30.922262 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00065f28-3393-4e02-8a8f-95ab91c100f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:09:30.922544 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00065f28-3393-4e02-8a8f-95ab91c100f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:09:30.922819 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00065f28-3393-4e02-8a8f-95ab91c100f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:09:30.923536 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00065f28-3393-4e02-8a8f-95ab91c100f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:09:30.923880 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00065f28-3393-4e02-8a8f-95ab91c100f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:09:30.924878 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: WARNING py.warnings [None req-00065f28-3393-4e02-8a8f-95ab91c100f3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 06 17:09:30.924878 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: warnings.warn(msg) Jul 06 17:09:30.924878 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: Jul 06 17:09:30.925744 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00065f28-3393-4e02-8a8f-95ab91c100f3 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:09:30.937420 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 762/1496] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:09:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:09:30.945178 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ad07c-43ae-4737-82db-94d3e9f913b6 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:09:30.955179 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96ad07c-43ae-4737-82db-94d3e9f913b6 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:09:30.959785 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73caf922-5539-4868-8119-26ace36b5f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:09:30.960447 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73caf922-5539-4868-8119-26ace36b5f94 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:09:30.960722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73caf922-5539-4868-8119-26ace36b5f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:09:30.961003 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73caf922-5539-4868-8119-26ace36b5f94 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:09:30.961699 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73caf922-5539-4868-8119-26ace36b5f94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:09:30.963434 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: WARNING py.warnings [None req-73caf922-5539-4868-8119-26ace36b5f94 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 06 17:09:30.963434 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: warnings.warn(msg) Jul 06 17:09:30.963434 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: Jul 06 17:09:30.964654 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73caf922-5539-4868-8119-26ace36b5f94 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:09:31.066898 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 735/1497] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:09:30 2022] POST /identity/v3/projects => generated 377 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:09:31.070922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00065f28-3393-4e02-8a8f-95ab91c100f3 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:09:31.081434 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00065f28-3393-4e02-8a8f-95ab91c100f3 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:09:31.137741 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f241916f-bc4a-483e-8e3e-d6d078df8995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:09:31.138412 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241916f-bc4a-483e-8e3e-d6d078df8995 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:09:31.138717 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241916f-bc4a-483e-8e3e-d6d078df8995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:09:31.138992 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f241916f-bc4a-483e-8e3e-d6d078df8995 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:09:31.139684 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f241916f-bc4a-483e-8e3e-d6d078df8995 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:09:31.140750 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: WARNING py.warnings [None req-f241916f-bc4a-483e-8e3e-d6d078df8995 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 06 17:09:31.140750 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: warnings.warn(msg) Jul 06 17:09:31.140750 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: Jul 06 17:09:31.141968 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f241916f-bc4a-483e-8e3e-d6d078df8995 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:09:31.214664 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 763/1498] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:09:31 2022] POST /identity/v3/users => generated 289 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:09:31.218848 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73caf922-5539-4868-8119-26ace36b5f94 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:09:31.228943 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73caf922-5539-4868-8119-26ace36b5f94 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:09:31.233960 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c9b8e1-46b8-435f-94bb-3569f7481a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:09:31.234615 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9b8e1-46b8-435f-94bb-3569f7481a98 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:09:31.234893 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9b8e1-46b8-435f-94bb-3569f7481a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:09:31.235163 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c9b8e1-46b8-435f-94bb-3569f7481a98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:09:31.235913 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c9b8e1-46b8-435f-94bb-3569f7481a98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:09:31.236947 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: WARNING py.warnings [None req-a0c9b8e1-46b8-435f-94bb-3569f7481a98 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 06 17:09:31.236947 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: warnings.warn(msg) Jul 06 17:09:31.236947 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: Jul 06 17:09:31.237829 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c9b8e1-46b8-435f-94bb-3569f7481a98 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:09:31.251035 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 736/1499] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:09:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:09:31.261897 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f241916f-bc4a-483e-8e3e-d6d078df8995 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:09:31.271663 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f241916f-bc4a-483e-8e3e-d6d078df8995 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:09:31.276269 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3390b3a-7fc6-4527-bd7b-d5b382732c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:09:31.277181 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3390b3a-7fc6-4527-bd7b-d5b382732c11 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:09:31.277465 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3390b3a-7fc6-4527-bd7b-d5b382732c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:09:31.277739 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3390b3a-7fc6-4527-bd7b-d5b382732c11 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:09:31.278459 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3390b3a-7fc6-4527-bd7b-d5b382732c11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:09:31.279461 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: WARNING py.warnings [None req-f3390b3a-7fc6-4527-bd7b-d5b382732c11 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 06 17:09:31.279461 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: warnings.warn(msg) Jul 06 17:09:31.279461 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: Jul 06 17:09:31.280358 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3390b3a-7fc6-4527-bd7b-d5b382732c11 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:09:31.293322 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 764/1500] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:09:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:09:31.297296 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c9b8e1-46b8-435f-94bb-3569f7481a98 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:09:31.306865 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c9b8e1-46b8-435f-94bb-3569f7481a98 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:09:31.311910 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2e6a87-3091-40f2-8070-73afc736b6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:09:31.312779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e6a87-3091-40f2-8070-73afc736b6e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:09:31.313058 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e6a87-3091-40f2-8070-73afc736b6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:09:31.313332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2e6a87-3091-40f2-8070-73afc736b6e9 admin admin] PATH_INFO: `/v3/projects/afba7b873b8b403987f14412aa162595/users/1484fb9f34a74d878929ba3bec79104b/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:09:31.327545 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2e6a87-3091-40f2-8070-73afc736b6e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afba7b873b8b403987f14412aa162595, user_id=1484fb9f34a74d878929ba3bec79104b, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:09:31.328539 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: WARNING py.warnings [None req-eb2e6a87-3091-40f2-8070-73afc736b6e9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 06 17:09:31.328539 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: warnings.warn(msg) Jul 06 17:09:31.328539 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: Jul 06 17:09:31.330440 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2e6a87-3091-40f2-8070-73afc736b6e9 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:09:31.397716 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 737/1501] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:09:31 2022] PUT /identity/v3/projects/afba7b873b8b403987f14412aa162595/users/1484fb9f34a74d878929ba3bec79104b/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:09:31.410415 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe0073-b3b7-42be-b874-072399b54216 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:09:31.410712 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe0073-b3b7-42be-b874-072399b54216 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:09:31.410998 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abe0073-b3b7-42be-b874-072399b54216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:09:31.437860 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-1abe0073-b3b7-42be-b874-072399b54216 None None] MFA Rules not processed for user `1484fb9f34a74d878929ba3bec79104b`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:09:31.439964 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-1abe0073-b3b7-42be-b874-072399b54216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:09:31.506783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 765/1502] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:09:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:09:31.519799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2e24c-5a93-4093-9f4d-e59adf855a78 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:09:31.520092 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2e24c-5a93-4093-9f4d-e59adf855a78 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:09:31.520373 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b2e24c-5a93-4093-9f4d-e59adf855a78 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:09:31.521726 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 738/1503] 10.222.0.40 () {54 vars in 963 bytes} [Wed Jul 6 17:09:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:09:31.526620 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a917b-d544-4331-b486-257d8beadd39 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:09:31.526950 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a917b-d544-4331-b486-257d8beadd39 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:09:31.527754 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15a917b-d544-4331-b486-257d8beadd39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:09:31.545742 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-c15a917b-d544-4331-b486-257d8beadd39 None None] MFA Rules not processed for user `a1bfa3e856ae46d48c82195f70aec24f`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:09:31.546962 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c15a917b-d544-4331-b486-257d8beadd39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:09:31.588821 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:09:31.596386 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 766/1504] 10.222.0.40 () {58 vars in 1060 bytes} [Wed Jul 6 17:09:31 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:09:31.603945 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b2e24c-5a93-4093-9f4d-e59adf855a78 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:09:31.615087 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b2e24c-5a93-4093-9f4d-e59adf855a78 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:09:31.618602 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ff572-ccf3-4685-80b5-3c9bb1142b9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:09:31.619281 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ff572-ccf3-4685-80b5-3c9bb1142b9e service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:09:31.619562 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ff572-ccf3-4685-80b5-3c9bb1142b9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:09:31.619936 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ff572-ccf3-4685-80b5-3c9bb1142b9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:09:31.628037 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964ff572-ccf3-4685-80b5-3c9bb1142b9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:09:31.629458 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964ff572-ccf3-4685-80b5-3c9bb1142b9e service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:09:31.638295 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 739/1505] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:09:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:09:31.680062 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4936909-da89-4f0a-9a70-1f6203e941a6 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:09:31.680387 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4936909-da89-4f0a-9a70-1f6203e941a6 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:09:31.680678 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4936909-da89-4f0a-9a70-1f6203e941a6 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:09:31.681959 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 767/1506] 10.222.0.40 () {52 vars in 829 bytes} [Wed Jul 6 17:09:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:09:31.685382 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ff572-ccf3-4685-80b5-3c9bb1142b9e service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:09:31.747105 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ff572-ccf3-4685-80b5-3c9bb1142b9e service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:09:31.751902 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746452e5-0324-47d8-9a77-81b189ca4c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:09:31.752783 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746452e5-0324-47d8-9a77-81b189ca4c31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:09:31.753066 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746452e5-0324-47d8-9a77-81b189ca4c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:09:31.753343 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746452e5-0324-47d8-9a77-81b189ca4c31 admin admin] PATH_INFO: `/v3/users/1484fb9f34a74d878929ba3bec79104b` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:09:31.754524 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746452e5-0324-47d8-9a77-81b189ca4c31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1484fb9f34a74d878929ba3bec79104b)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:09:31.755532 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: WARNING py.warnings [None req-746452e5-0324-47d8-9a77-81b189ca4c31 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 06 17:09:31.755532 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: warnings.warn(msg) Jul 06 17:09:31.755532 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: Jul 06 17:09:31.756818 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746452e5-0324-47d8-9a77-81b189ca4c31 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:09:31.957364 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-746452e5-0324-47d8-9a77-81b189ca4c31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1484fb9f34a74d878929ba3bec79104b'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:09:31.975339 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-746452e5-0324-47d8-9a77-81b189ca4c31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1484fb9f34a74d878929ba3bec79104b'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:09:31.987933 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-746452e5-0324-47d8-9a77-81b189ca4c31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1484fb9f34a74d878929ba3bec79104b'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:09:31.988717 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-746452e5-0324-47d8-9a77-81b189ca4c31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1484fb9f34a74d878929ba3bec79104b'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:09:31.992690 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:09:32.094457 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-746452e5-0324-47d8-9a77-81b189ca4c31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1484fb9f34a74d878929ba3bec79104b'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:09:32.100697 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 740/1507] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:09:31 2022] DELETE /identity/v3/users/1484fb9f34a74d878929ba3bec79104b => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:09:32.105073 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80bbd0-8519-4edb-bb3e-f3f5b4c92f3a None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:09:32.106038 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80bbd0-8519-4edb-bb3e-f3f5b4c92f3a None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:09:32.106184 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80bbd0-8519-4edb-bb3e-f3f5b4c92f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:09:32.123781 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-ad80bbd0-8519-4edb-bb3e-f3f5b4c92f3a None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:09:32.124972 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-ad80bbd0-8519-4edb-bb3e-f3f5b4c92f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:09:32.181424 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 768/1508] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:09:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:09:32.191386 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746452e5-0324-47d8-9a77-81b189ca4c31 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:09:32.192809 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-746452e5-0324-47d8-9a77-81b189ca4c31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:09:32.248638 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746452e5-0324-47d8-9a77-81b189ca4c31 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:09:32.252169 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b428d5-679e-4fc7-b8a9-c6408249c9e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:09:32.252814 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b428d5-679e-4fc7-b8a9-c6408249c9e1 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:09:32.253101 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b428d5-679e-4fc7-b8a9-c6408249c9e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:09:32.253379 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b428d5-679e-4fc7-b8a9-c6408249c9e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:09:32.262184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b428d5-679e-4fc7-b8a9-c6408249c9e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:09:32.263589 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b428d5-679e-4fc7-b8a9-c6408249c9e1 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:09:32.272830 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 741/1509] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:09:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 17:09:32.713943 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80bbd0-8519-4edb-bb3e-f3f5b4c92f3a None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:09:32.715976 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-ad80bbd0-8519-4edb-bb3e-f3f5b4c92f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:09:32.728685 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80bbd0-8519-4edb-bb3e-f3f5b4c92f3a None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:09:32.737047 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee3eecf-3614-49df-b35f-b6e43ec4d4ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:09:32.738052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3eecf-3614-49df-b35f-b6e43ec4d4ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:09:32.738403 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3eecf-3614-49df-b35f-b6e43ec4d4ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:09:32.738691 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee3eecf-3614-49df-b35f-b6e43ec4d4ae admin admin] PATH_INFO: `/v3/projects/afba7b873b8b403987f14412aa162595` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:09:32.740151 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3eecf-3614-49df-b35f-b6e43ec4d4ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=afba7b873b8b403987f14412aa162595)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:09:32.741180 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: WARNING py.warnings [None req-5ee3eecf-3614-49df-b35f-b6e43ec4d4ae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 06 17:09:32.741180 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: warnings.warn(msg) Jul 06 17:09:32.741180 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: Jul 06 17:09:32.742578 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee3eecf-3614-49df-b35f-b6e43ec4d4ae admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:09:32.866092 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-5ee3eecf-3614-49df-b35f-b6e43ec4d4ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afba7b873b8b403987f14412aa162595', 'request_id': 'req-5ee3eecf-3614-49df-b35f-b6e43ec4d4ae'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:09:32.878896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-5ee3eecf-3614-49df-b35f-b6e43ec4d4ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afba7b873b8b403987f14412aa162595', 'request_id': 'req-5ee3eecf-3614-49df-b35f-b6e43ec4d4ae'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:09:32.885610 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-5ee3eecf-3614-49df-b35f-b6e43ec4d4ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afba7b873b8b403987f14412aa162595', 'request_id': 'req-5ee3eecf-3614-49df-b35f-b6e43ec4d4ae'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:09:32.951452 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-5ee3eecf-3614-49df-b35f-b6e43ec4d4ae admin admin] The token cache is being invalidate because project afba7b873b8b403987f14412aa162595 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:09:32.951757 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-5ee3eecf-3614-49df-b35f-b6e43ec4d4ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:09:32.952477 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-5ee3eecf-3614-49df-b35f-b6e43ec4d4ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:09:32.954075 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 769/1510] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:09:32 2022] DELETE /identity/v3/projects/afba7b873b8b403987f14412aa162595 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:09:42.268745 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:09:42.947915 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:11:10.047936 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e7378-8553-4676-9d26-67a8899d07de None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:11:10.048496 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e7378-8553-4676-9d26-67a8899d07de None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:11:10.048496 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17e7378-8553-4676-9d26-67a8899d07de None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:11:10.049891 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 742/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 6 17:11:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:11:10.054799 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39faecc-09a2-470c-a1a2-689eb80c7af0 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:11:10.055252 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39faecc-09a2-470c-a1a2-689eb80c7af0 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:11:10.055355 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39faecc-09a2-470c-a1a2-689eb80c7af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:11:10.072966 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-c39faecc-09a2-470c-a1a2-689eb80c7af0 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:11:10.074155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c39faecc-09a2-470c-a1a2-689eb80c7af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:11:10.134138 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 770/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 6 17:11:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:11:10.289510 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17e7378-8553-4676-9d26-67a8899d07de None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:11:10.291262 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e17e7378-8553-4676-9d26-67a8899d07de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:11:10.364821 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17e7378-8553-4676-9d26-67a8899d07de None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:11:10.370486 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddef40af-a946-4f3d-9bb9-de3f1e62534b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:11:10.371177 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef40af-a946-4f3d-9bb9-de3f1e62534b service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:11:10.371488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef40af-a946-4f3d-9bb9-de3f1e62534b service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:11:10.371795 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddef40af-a946-4f3d-9bb9-de3f1e62534b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:11:10.380581 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddef40af-a946-4f3d-9bb9-de3f1e62534b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:11:10.382349 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddef40af-a946-4f3d-9bb9-de3f1e62534b service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:11:10.392184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 743/1513] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:11:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:11:11.221299 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd302c2-9e4b-4128-a250-1b6cb404c3f6 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:11:11.221778 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd302c2-9e4b-4128-a250-1b6cb404c3f6 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:11:11.221894 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd302c2-9e4b-4128-a250-1b6cb404c3f6 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:11:11.223213 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 771/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 6 17:11:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:11:11.227905 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca421b-805a-4f36-9c3e-e46f4c5768a6 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:11:11.228350 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca421b-805a-4f36-9c3e-e46f4c5768a6 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:11:11.228490 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bca421b-805a-4f36-9c3e-e46f4c5768a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:11:11.245904 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-4bca421b-805a-4f36-9c3e-e46f4c5768a6 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:11:11.247095 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4bca421b-805a-4f36-9c3e-e46f4c5768a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:11:11.255244 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 744/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 6 17:11:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:11:11.406539 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd302c2-9e4b-4128-a250-1b6cb404c3f6 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:11:11.420824 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd302c2-9e4b-4128-a250-1b6cb404c3f6 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:11:11.426666 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf703c-963f-47b2-91e9-87b11bdb2d15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:11:11.427303 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf703c-963f-47b2-91e9-87b11bdb2d15 service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:11:11.427588 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf703c-963f-47b2-91e9-87b11bdb2d15 service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:11:11.427900 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf703c-963f-47b2-91e9-87b11bdb2d15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:11:11.435872 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabf703c-963f-47b2-91e9-87b11bdb2d15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:11:11.437451 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabf703c-963f-47b2-91e9-87b11bdb2d15 service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:11:11.447567 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 772/1516] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:11:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:11:12.026008 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e8cd5d-9ddd-40a9-811d-eba8bb38ad15 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:11:12.026320 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e8cd5d-9ddd-40a9-811d-eba8bb38ad15 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:11:12.026861 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e8cd5d-9ddd-40a9-811d-eba8bb38ad15 None None] PATH_INFO: `/v3` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:11:12.028217 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 745/1517] 10.222.0.50 () {54 vars in 925 bytes} [Wed Jul 6 17:11:12 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:11:12.034924 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7df03-271b-4333-8c81-971c3b84eabc None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:11:12.035221 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7df03-271b-4333-8c81-971c3b84eabc None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:11:12.035531 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a7df03-271b-4333-8c81-971c3b84eabc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:11:12.054218 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-e9a7df03-271b-4333-8c81-971c3b84eabc None None] MFA Rules not processed for user `76ca877bee9041d28ad41e45ef3dd18c`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:11:12.055429 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-e9a7df03-271b-4333-8c81-971c3b84eabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:11:12.116152 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 773/1518] 10.222.0.50 () {58 vars in 1015 bytes} [Wed Jul 6 17:11:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:11:16.040472 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9444a3-28f6-49cc-99a7-44ccce36f66d None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:11:16.041008 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9444a3-28f6-49cc-99a7-44ccce36f66d None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:11:16.041008 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9444a3-28f6-49cc-99a7-44ccce36f66d None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:11:16.042324 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 746/1519] 10.222.0.40 () {54 vars in 960 bytes} [Wed Jul 6 17:11:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:11:16.047428 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cdb960-7428-4b34-8289-909a0be67489 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:11:16.048441 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cdb960-7428-4b34-8289-909a0be67489 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:11:16.048566 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cdb960-7428-4b34-8289-909a0be67489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:11:16.066808 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-a8cdb960-7428-4b34-8289-909a0be67489 None None] MFA Rules not processed for user `76ca877bee9041d28ad41e45ef3dd18c`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:11:16.068030 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-a8cdb960-7428-4b34-8289-909a0be67489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:11:16.076484 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 774/1520] 10.222.0.40 () {58 vars in 1057 bytes} [Wed Jul 6 17:11:16 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:11:16.083835 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9444a3-28f6-49cc-99a7-44ccce36f66d None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:11:16.096920 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9444a3-28f6-49cc-99a7-44ccce36f66d None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:11:16.101736 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39588cac-8d8d-487f-8278-682677c1b2e9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76ca877bee9041d28ad41e45ef3dd18c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:11:16.102355 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39588cac-8d8d-487f-8278-682677c1b2e9 service placement] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:11:16.102673 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39588cac-8d8d-487f-8278-682677c1b2e9 service placement] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:11:16.102949 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39588cac-8d8d-487f-8278-682677c1b2e9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:11:16.111618 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39588cac-8d8d-487f-8278-682677c1b2e9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:11:16.113273 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39588cac-8d8d-487f-8278-682677c1b2e9 service placement] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:11:16.122691 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 747/1521] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:11:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 17:11:22.676899 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceae514-3752-4cac-a1fb-3e034bd30a88 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:11:22.677477 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceae514-3752-4cac-a1fb-3e034bd30a88 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:11:22.677477 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceae514-3752-4cac-a1fb-3e034bd30a88 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:11:22.678869 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 775/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 6 17:11:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:11:22.683607 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f6d88-216a-44cd-a4c0-d6eccd34febe None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:11:22.684072 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f6d88-216a-44cd-a4c0-d6eccd34febe None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:11:22.684204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381f6d88-216a-44cd-a4c0-d6eccd34febe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:11:22.702282 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-381f6d88-216a-44cd-a4c0-d6eccd34febe None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:11:22.703451 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-381f6d88-216a-44cd-a4c0-d6eccd34febe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:11:22.711837 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 748/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 6 17:11:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:11:22.865690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceae514-3752-4cac-a1fb-3e034bd30a88 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:11:22.879988 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceae514-3752-4cac-a1fb-3e034bd30a88 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:11:22.885695 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f1f1f1-c56b-419a-a8c5-a9674ec15e68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:11:22.886330 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f1f1f1-c56b-419a-a8c5-a9674ec15e68 service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:11:22.886611 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f1f1f1-c56b-419a-a8c5-a9674ec15e68 service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:11:22.886884 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f1f1f1-c56b-419a-a8c5-a9674ec15e68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:11:22.896719 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f1f1f1-c56b-419a-a8c5-a9674ec15e68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:11:22.898343 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f1f1f1-c56b-419a-a8c5-a9674ec15e68 service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:11:22.910185 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 776/1524] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:11:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:11:23.715304 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee937dcd-e3b9-4d8d-b6ec-8ff63f573337 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:11:23.715864 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee937dcd-e3b9-4d8d-b6ec-8ff63f573337 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:11:23.715982 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee937dcd-e3b9-4d8d-b6ec-8ff63f573337 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:11:23.717286 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 749/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 6 17:11:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:11:23.722754 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d14b352-59f4-466c-b7d6-7ef836c6585d None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:11:23.723188 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d14b352-59f4-466c-b7d6-7ef836c6585d None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:11:23.723285 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d14b352-59f4-466c-b7d6-7ef836c6585d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:11:23.741672 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-3d14b352-59f4-466c-b7d6-7ef836c6585d None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:11:23.742852 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-3d14b352-59f4-466c-b7d6-7ef836c6585d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:11:23.750493 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 777/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 6 17:11:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:11:23.901363 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee937dcd-e3b9-4d8d-b6ec-8ff63f573337 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:11:23.916585 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee937dcd-e3b9-4d8d-b6ec-8ff63f573337 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:11:23.921938 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd20a1-dce7-485d-80e2-24a12872eb0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:11:23.922577 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd20a1-dce7-485d-80e2-24a12872eb0f service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:11:23.922865 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd20a1-dce7-485d-80e2-24a12872eb0f service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:11:23.923151 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd20a1-dce7-485d-80e2-24a12872eb0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:11:23.933164 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffd20a1-dce7-485d-80e2-24a12872eb0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:11:23.934947 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffd20a1-dce7-485d-80e2-24a12872eb0f service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:11:23.945851 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 750/1527] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:11:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:11:27.772716 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9add62-1dfc-4d99-b139-2a06a19ec33b None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:11:27.773170 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9add62-1dfc-4d99-b139-2a06a19ec33b None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:11:27.773318 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9add62-1dfc-4d99-b139-2a06a19ec33b None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:11:27.774620 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 778/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 17:11:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:11:27.780341 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d441ed-dbe6-42f4-a3d6-f4dabaa631dc None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:11:27.780766 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d441ed-dbe6-42f4-a3d6-f4dabaa631dc None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:11:27.780912 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d441ed-dbe6-42f4-a3d6-f4dabaa631dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:11:27.801761 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-53d441ed-dbe6-42f4-a3d6-f4dabaa631dc None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:11:27.802971 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-53d441ed-dbe6-42f4-a3d6-f4dabaa631dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:11:27.810777 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 751/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 17:11:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:11:27.890999 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9add62-1dfc-4d99-b139-2a06a19ec33b None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:11:27.892788 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-8c9add62-1dfc-4d99-b139-2a06a19ec33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:11:27.951195 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9add62-1dfc-4d99-b139-2a06a19ec33b None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:11:27.954733 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d38c0f4-552c-4a7a-897a-6379fbf6be19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:11:27.955379 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d38c0f4-552c-4a7a-897a-6379fbf6be19 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:11:27.956476 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d38c0f4-552c-4a7a-897a-6379fbf6be19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:11:27.956760 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d38c0f4-552c-4a7a-897a-6379fbf6be19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:11:27.965573 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d38c0f4-552c-4a7a-897a-6379fbf6be19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:11:27.967006 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d38c0f4-552c-4a7a-897a-6379fbf6be19 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:11:27.972550 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:11:27.977221 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 779/1530] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:11:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:11:30.028955 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55452c35-8329-4962-b1ac-820758ba2d24 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:11:30.029523 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55452c35-8329-4962-b1ac-820758ba2d24 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:11:30.029523 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55452c35-8329-4962-b1ac-820758ba2d24 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:11:30.030818 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 752/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 17:11:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:11:30.036038 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a629c66-1a64-42a8-807d-87cc73c47973 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:11:30.036431 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a629c66-1a64-42a8-807d-87cc73c47973 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:11:30.036600 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a629c66-1a64-42a8-807d-87cc73c47973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:11:30.054695 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-7a629c66-1a64-42a8-807d-87cc73c47973 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:11:30.055909 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-7a629c66-1a64-42a8-807d-87cc73c47973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:11:30.063587 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 780/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 17:11:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:11:30.098777 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55452c35-8329-4962-b1ac-820758ba2d24 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:11:30.109434 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55452c35-8329-4962-b1ac-820758ba2d24 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:11:30.113481 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea3c56-b881-4fbb-9f53-310b6defba67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:11:30.114113 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea3c56-b881-4fbb-9f53-310b6defba67 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:11:30.114400 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea3c56-b881-4fbb-9f53-310b6defba67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:11:30.114681 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea3c56-b881-4fbb-9f53-310b6defba67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:11:30.123444 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ea3c56-b881-4fbb-9f53-310b6defba67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:11:30.124904 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ea3c56-b881-4fbb-9f53-310b6defba67 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:11:30.134820 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 753/1533] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:11:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:11:33.696663 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f15554-ecd3-4de2-9679-ea1105dac20a None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:11:33.697255 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f15554-ecd3-4de2-9679-ea1105dac20a None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:11:33.697255 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f15554-ecd3-4de2-9679-ea1105dac20a None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:11:33.698598 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 781/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 17:11:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:11:33.703670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22926f62-632d-4e8f-b68e-5be117b28f38 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:11:33.704111 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22926f62-632d-4e8f-b68e-5be117b28f38 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:11:33.704259 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22926f62-632d-4e8f-b68e-5be117b28f38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:11:33.711071 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=62 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:11:33.841310 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-22926f62-632d-4e8f-b68e-5be117b28f38 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:11:33.842691 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-22926f62-632d-4e8f-b68e-5be117b28f38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:11:33.850611 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 754/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 17:11:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:11:34.150669 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f15554-ecd3-4de2-9679-ea1105dac20a None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:11:34.152662 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-51f15554-ecd3-4de2-9679-ea1105dac20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:11:34.208947 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f15554-ecd3-4de2-9679-ea1105dac20a None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:11:34.212496 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaea9c2-6c20-417b-87e8-fad16034dab0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:11:34.213137 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaea9c2-6c20-417b-87e8-fad16034dab0 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:11:34.213424 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaea9c2-6c20-417b-87e8-fad16034dab0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:11:34.213701 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaea9c2-6c20-417b-87e8-fad16034dab0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:11:34.223037 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaea9c2-6c20-417b-87e8-fad16034dab0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:11:34.224485 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaea9c2-6c20-417b-87e8-fad16034dab0 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:11:34.233713 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 782/1536] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:11:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:11:36.261211 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ae54f-4728-4fb3-a91f-6465732e74af None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:11:36.261757 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ae54f-4728-4fb3-a91f-6465732e74af None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:11:36.261757 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9ae54f-4728-4fb3-a91f-6465732e74af None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:11:36.263147 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 755/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 6 17:11:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:11:36.268847 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f23d1c5-3f38-49ff-9ca8-5ca13d6b0e74 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:11:36.269291 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f23d1c5-3f38-49ff-9ca8-5ca13d6b0e74 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:11:36.269396 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f23d1c5-3f38-49ff-9ca8-5ca13d6b0e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:11:36.287822 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-7f23d1c5-3f38-49ff-9ca8-5ca13d6b0e74 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:11:36.288996 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-7f23d1c5-3f38-49ff-9ca8-5ca13d6b0e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:11:36.296634 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 783/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 6 17:11:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:11:36.593148 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9ae54f-4728-4fb3-a91f-6465732e74af None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:11:36.594797 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-2c9ae54f-4728-4fb3-a91f-6465732e74af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:11:36.607372 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9ae54f-4728-4fb3-a91f-6465732e74af None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:11:36.610818 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96590ca-864e-465a-b411-6eb39865423a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:11:36.611570 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96590ca-864e-465a-b411-6eb39865423a service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:11:36.611891 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96590ca-864e-465a-b411-6eb39865423a service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:11:36.612172 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96590ca-864e-465a-b411-6eb39865423a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:11:36.620839 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96590ca-864e-465a-b411-6eb39865423a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:11:36.622277 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96590ca-864e-465a-b411-6eb39865423a service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:11:36.631983 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 756/1539] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:11:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:11:46.282198 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=32 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:11:46.627383 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=8 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:04.154838 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4217bee-0d27-4fc1-b912-405e6e3e3806 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:04.155487 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4217bee-0d27-4fc1-b912-405e6e3e3806 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:04.155487 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4217bee-0d27-4fc1-b912-405e6e3e3806 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:04.181467 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-e4217bee-0d27-4fc1-b912-405e6e3e3806 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:04.182671 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-e4217bee-0d27-4fc1-b912-405e6e3e3806 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:04.194609 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 784/1540] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:04.195613 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5faf4a-78ff-4d11-a8f8-c3982e7d8616 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:04.196846 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5faf4a-78ff-4d11-a8f8-c3982e7d8616 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:04.196846 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5faf4a-78ff-4d11-a8f8-c3982e7d8616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:04.202828 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2f4b9-465c-4ab6-ae8b-b806ffccc670 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:04.203156 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2f4b9-465c-4ab6-ae8b-b806ffccc670 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:04.203441 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a2f4b9-465c-4ab6-ae8b-b806ffccc670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:04.222096 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-ce5faf4a-78ff-4d11-a8f8-c3982e7d8616 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:04.232755 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ce5faf4a-78ff-4d11-a8f8-c3982e7d8616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:04.233401 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-35a2f4b9-465c-4ab6-ae8b-b806ffccc670 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:04.234776 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-35a2f4b9-465c-4ab6-ae8b-b806ffccc670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:04.237741 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 757/1541] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:04.247181 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6772897-f6c2-4e58-bd7f-dc34b2c24693 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:04.247480 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6772897-f6c2-4e58-bd7f-dc34b2c24693 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:04.247785 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6772897-f6c2-4e58-bd7f-dc34b2c24693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:04.248654 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 785/1542] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:13:04.253835 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a2f4b9-465c-4ab6-ae8b-b806ffccc670 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:04.267827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a2f4b9-465c-4ab6-ae8b-b806ffccc670 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:04.275667 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e484077d-6c2e-4df5-a15f-be24c655bace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:04.276129 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-a6772897-f6c2-4e58-bd7f-dc34b2c24693 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:04.276614 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e484077d-6c2e-4df5-a15f-be24c655bace admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:04.276958 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e484077d-6c2e-4df5-a15f-be24c655bace admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:04.277242 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e484077d-6c2e-4df5-a15f-be24c655bace admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:04.277603 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-a6772897-f6c2-4e58-bd7f-dc34b2c24693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:04.277965 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e484077d-6c2e-4df5-a15f-be24c655bace admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:13:04.278270 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e484077d-6c2e-4df5-a15f-be24c655bace admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:04.279881 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e484077d-6c2e-4df5-a15f-be24c655bace admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:04.294863 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 758/1543] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:13:04.297381 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae3e6a9-89c4-4337-90c2-b65d92bceafc None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:04.298106 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae3e6a9-89c4-4337-90c2-b65d92bceafc None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:04.300411 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 786/1544] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:13:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:13:04.303978 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae3e6a9-89c4-4337-90c2-b65d92bceafc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:04.306645 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c7a11-f6f8-448d-a5ef-ce12763423f7 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:04.306939 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c7a11-f6f8-448d-a5ef-ce12763423f7 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:04.307215 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c7a11-f6f8-448d-a5ef-ce12763423f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:04.346081 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-fae3e6a9-89c4-4337-90c2-b65d92bceafc None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:04.347295 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-fae3e6a9-89c4-4337-90c2-b65d92bceafc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:04.360290 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-005c7a11-f6f8-448d-a5ef-ce12763423f7 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:04.361492 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-005c7a11-f6f8-448d-a5ef-ce12763423f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:04.376603 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 759/1545] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:04.377516 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae3e6a9-89c4-4337-90c2-b65d92bceafc None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:04.380613 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 787/1546] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:04.381583 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c7a11-f6f8-448d-a5ef-ce12763423f7 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:04.395952 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae3e6a9-89c4-4337-90c2-b65d92bceafc None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:04.403170 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c7a11-f6f8-448d-a5ef-ce12763423f7 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:04.411692 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e547d38-415d-4108-837a-7b7c8915f169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:04.412439 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e547d38-415d-4108-837a-7b7c8915f169 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:04.412821 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e547d38-415d-4108-837a-7b7c8915f169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:04.413101 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e547d38-415d-4108-837a-7b7c8915f169 admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:04.413836 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e547d38-415d-4108-837a-7b7c8915f169 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:13:04.414141 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e547d38-415d-4108-837a-7b7c8915f169 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:04.415274 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: WARNING py.warnings [None req-5e547d38-415d-4108-837a-7b7c8915f169 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 06 17:13:04.415274 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: warnings.warn(msg) Jul 06 17:13:04.415274 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: Jul 06 17:13:04.415573 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e8eca2-3ec0-41f3-a946-02db869a3d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:04.416208 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e547d38-415d-4108-837a-7b7c8915f169 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:04.416334 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8eca2-3ec0-41f3-a946-02db869a3d24 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:04.416575 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8eca2-3ec0-41f3-a946-02db869a3d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:04.416859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8eca2-3ec0-41f3-a946-02db869a3d24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:04.417566 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e8eca2-3ec0-41f3-a946-02db869a3d24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:04.418859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: WARNING py.warnings [None req-99e8eca2-3ec0-41f3-a946-02db869a3d24 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 06 17:13:04.418859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: warnings.warn(msg) Jul 06 17:13:04.418859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: Jul 06 17:13:04.420690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e8eca2-3ec0-41f3-a946-02db869a3d24 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:04.430346 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 760/1547] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:13:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:13:04.431913 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaff060-948b-4b0d-ab50-85e77786708c None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:04.432271 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaff060-948b-4b0d-ab50-85e77786708c None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:04.432617 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaff060-948b-4b0d-ab50-85e77786708c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:04.451528 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-8eaff060-948b-4b0d-ab50-85e77786708c None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:04.452748 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-8eaff060-948b-4b0d-ab50-85e77786708c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:04.461112 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 761/1548] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:04.462432 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f944725-a8f9-4c35-9a09-27a0cf770b6b None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:04.462717 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f944725-a8f9-4c35-9a09-27a0cf770b6b None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:04.462986 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f944725-a8f9-4c35-9a09-27a0cf770b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:04.480651 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-3f944725-a8f9-4c35-9a09-27a0cf770b6b None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:04.481897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-3f944725-a8f9-4c35-9a09-27a0cf770b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:04.490734 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 762/1549] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:04.492063 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badeee3a-041e-48d3-b084-20d3e9761784 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:04.492357 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badeee3a-041e-48d3-b084-20d3e9761784 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:04.492635 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badeee3a-041e-48d3-b084-20d3e9761784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:04.508080 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 788/1550] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:13:04 2022] POST /identity/v3/projects => generated 403 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:04.509520 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c7939-d5a9-4a48-b871-16e7857982ed None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:04.509815 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c7939-d5a9-4a48-b871-16e7857982ed None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:04.510093 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5c7939-d5a9-4a48-b871-16e7857982ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:04.510642 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-badeee3a-041e-48d3-b084-20d3e9761784 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:04.511859 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-badeee3a-041e-48d3-b084-20d3e9761784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:04.529532 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-6c5c7939-d5a9-4a48-b871-16e7857982ed None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:04.530719 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-6c5c7939-d5a9-4a48-b871-16e7857982ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:04.570797 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 763/1551] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:04.572281 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bfe182-ef5e-48ad-8d2e-0be6e91e266a None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:04.573199 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bfe182-ef5e-48ad-8d2e-0be6e91e266a None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:04.573492 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bfe182-ef5e-48ad-8d2e-0be6e91e266a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:04.591245 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-d6bfe182-ef5e-48ad-8d2e-0be6e91e266a None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:04.592128 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 789/1552] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:04.592457 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d6bfe182-ef5e-48ad-8d2e-0be6e91e266a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:04.592940 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c7939-d5a9-4a48-b871-16e7857982ed None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:04.600562 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 764/1553] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:04.601910 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a882952-412f-46a1-bda9-e9fa2f5d60b4 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:04.602045 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5c7939-d5a9-4a48-b871-16e7857982ed None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:04.602222 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a882952-412f-46a1-bda9-e9fa2f5d60b4 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:04.602463 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a882952-412f-46a1-bda9-e9fa2f5d60b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:04.611162 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c162e4d-210c-4657-820b-7776f1e244e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:04.611862 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c162e4d-210c-4657-820b-7776f1e244e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:04.612150 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c162e4d-210c-4657-820b-7776f1e244e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:04.612427 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c162e4d-210c-4657-820b-7776f1e244e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:04.613151 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c162e4d-210c-4657-820b-7776f1e244e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:04.615052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c162e4d-210c-4657-820b-7776f1e244e5 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:04.620315 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-7a882952-412f-46a1-bda9-e9fa2f5d60b4 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:04.621533 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-7a882952-412f-46a1-bda9-e9fa2f5d60b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:04.630221 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 765/1554] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:04.631211 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d1827-c776-4aed-a9ad-c19a496bfe80 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:04.631498 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d1827-c776-4aed-a9ad-c19a496bfe80 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:04.631804 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81d1827-c776-4aed-a9ad-c19a496bfe80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:04.648846 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-e81d1827-c776-4aed-a9ad-c19a496bfe80 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:04.652149 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e81d1827-c776-4aed-a9ad-c19a496bfe80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:04.658277 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 766/1555] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:04.659153 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d1827-c776-4aed-a9ad-c19a496bfe80 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:04.668087 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81d1827-c776-4aed-a9ad-c19a496bfe80 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:04.673012 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20471ecd-7feb-4d44-a26a-b112d8fb7ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:04.673633 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20471ecd-7feb-4d44-a26a-b112d8fb7ff0 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:04.673916 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20471ecd-7feb-4d44-a26a-b112d8fb7ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:04.674187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20471ecd-7feb-4d44-a26a-b112d8fb7ff0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:04.674892 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20471ecd-7feb-4d44-a26a-b112d8fb7ff0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:13:04.675194 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20471ecd-7feb-4d44-a26a-b112d8fb7ff0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:04.676936 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20471ecd-7feb-4d44-a26a-b112d8fb7ff0 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:04.689179 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 767/1556] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:13:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:13:04.689927 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20471ecd-7feb-4d44-a26a-b112d8fb7ff0 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:04.699655 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20471ecd-7feb-4d44-a26a-b112d8fb7ff0 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:04.704252 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb43fed-a586-4382-b1e1-ac6943c8d83f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:04.704898 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb43fed-a586-4382-b1e1-ac6943c8d83f admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:04.705186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb43fed-a586-4382-b1e1-ac6943c8d83f admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:04.705466 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb43fed-a586-4382-b1e1-ac6943c8d83f admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:04.706149 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb43fed-a586-4382-b1e1-ac6943c8d83f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:04.707151 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: WARNING py.warnings [None req-ecb43fed-a586-4382-b1e1-ac6943c8d83f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 06 17:13:04.707151 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: warnings.warn(msg) Jul 06 17:13:04.707151 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: Jul 06 17:13:04.711332 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 790/1557] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:13:04 2022] POST /identity/v3/projects => generated 353 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:04.711500 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb43fed-a586-4382-b1e1-ac6943c8d83f admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:04.711603 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1387d121-a314-4f8d-9ef5-b385e0cb605f None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:04.711603 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1387d121-a314-4f8d-9ef5-b385e0cb605f None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:04.711603 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1387d121-a314-4f8d-9ef5-b385e0cb605f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:04.732283 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-1387d121-a314-4f8d-9ef5-b385e0cb605f None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:04.733672 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-1387d121-a314-4f8d-9ef5-b385e0cb605f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:04.787023 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 768/1558] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:13:04 2022] POST /identity/v3/users => generated 302 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:04.787974 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb43fed-a586-4382-b1e1-ac6943c8d83f admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:04.796737 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 791/1559] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:04.798544 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2014c8-6b31-4c31-bea5-bb2e0d21ba0b None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:04.798834 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2014c8-6b31-4c31-bea5-bb2e0d21ba0b None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:04.799110 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2014c8-6b31-4c31-bea5-bb2e0d21ba0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:04.800658 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb43fed-a586-4382-b1e1-ac6943c8d83f admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:04.805224 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3d0927-8212-4235-8f1a-b300c73b8162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:04.805916 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3d0927-8212-4235-8f1a-b300c73b8162 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:04.806225 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3d0927-8212-4235-8f1a-b300c73b8162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:04.806478 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3d0927-8212-4235-8f1a-b300c73b8162 admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:04.807215 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3d0927-8212-4235-8f1a-b300c73b8162 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:13:04.807524 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3d0927-8212-4235-8f1a-b300c73b8162 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:04.809223 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3d0927-8212-4235-8f1a-b300c73b8162 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:04.818210 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-bb2014c8-6b31-4c31-bea5-bb2e0d21ba0b None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:04.819862 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-bb2014c8-6b31-4c31-bea5-bb2e0d21ba0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:04.821978 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 769/1560] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:13:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:13:04.823073 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaca705-c450-451a-8c93-646561eb3a55 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:04.823367 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaca705-c450-451a-8c93-646561eb3a55 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:04.823803 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaca705-c450-451a-8c93-646561eb3a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:04.834455 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 792/1561] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:04.835275 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2014c8-6b31-4c31-bea5-bb2e0d21ba0b None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:04.844179 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-5eaca705-c450-451a-8c93-646561eb3a55 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:04.845304 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2014c8-6b31-4c31-bea5-bb2e0d21ba0b None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:04.845930 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-5eaca705-c450-451a-8c93-646561eb3a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:04.849864 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d5b30-6a00-4ac1-a5ad-6e7be4e364d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:04.850508 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d5b30-6a00-4ac1-a5ad-6e7be4e364d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:04.854664 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d5b30-6a00-4ac1-a5ad-6e7be4e364d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:04.854664 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d5b30-6a00-4ac1-a5ad-6e7be4e364d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:04.854664 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7d5b30-6a00-4ac1-a5ad-6e7be4e364d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:13:04.854664 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7d5b30-6a00-4ac1-a5ad-6e7be4e364d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:04.854664 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7d5b30-6a00-4ac1-a5ad-6e7be4e364d5 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:04.855082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 770/1562] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:04.855082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaca705-c450-451a-8c93-646561eb3a55 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:04.873827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 793/1563] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:13:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:13:04.873827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d5b30-6a00-4ac1-a5ad-6e7be4e364d5 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:04.874085 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaca705-c450-451a-8c93-646561eb3a55 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:04.874201 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:04.877589 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d5b30-6a00-4ac1-a5ad-6e7be4e364d5 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:04.878863 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c4a4f-5b62-47c4-adf3-a56432d7cc54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:04.879534 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c4a4f-5b62-47c4-adf3-a56432d7cc54 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:04.879859 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c4a4f-5b62-47c4-adf3-a56432d7cc54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:04.880136 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c4a4f-5b62-47c4-adf3-a56432d7cc54 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:04.880824 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93c4a4f-5b62-47c4-adf3-a56432d7cc54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:04.882493 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249373f5-0900-4c52-a318-e10f310c4f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:04.882881 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93c4a4f-5b62-47c4-adf3-a56432d7cc54 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:04.883135 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249373f5-0900-4c52-a318-e10f310c4f10 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:04.883423 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249373f5-0900-4c52-a318-e10f310c4f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:04.883699 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249373f5-0900-4c52-a318-e10f310c4f10 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:04.899274 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249373f5-0900-4c52-a318-e10f310c4f10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:04.899274 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249373f5-0900-4c52-a318-e10f310c4f10 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:05.050426 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 794/1564] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:13:04 2022] POST /identity/v3/users => generated 277 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:05.051171 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249373f5-0900-4c52-a318-e10f310c4f10 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:05.061191 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249373f5-0900-4c52-a318-e10f310c4f10 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:05.065765 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a1072-33a7-406a-8e0a-4513574aff20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:05.066403 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a1072-33a7-406a-8e0a-4513574aff20 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:05.066686 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a1072-33a7-406a-8e0a-4513574aff20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:05.066964 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323a1072-33a7-406a-8e0a-4513574aff20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:05.067675 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323a1072-33a7-406a-8e0a-4513574aff20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:05.069278 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323a1072-33a7-406a-8e0a-4513574aff20 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:05.079008 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 795/1565] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:05 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:05.083001 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a1072-33a7-406a-8e0a-4513574aff20 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:05.085456 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 771/1566] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:13:04 2022] POST /identity/v3/projects => generated 385 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:05.086319 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c4a4f-5b62-47c4-adf3-a56432d7cc54 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:05.090328 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323a1072-33a7-406a-8e0a-4513574aff20 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:05.114263 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:05.145465 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf1f616-0794-4a1d-bef2-dbf6521a0c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:05.146133 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf1f616-0794-4a1d-bef2-dbf6521a0c02 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:05.146413 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf1f616-0794-4a1d-bef2-dbf6521a0c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:05.146692 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf1f616-0794-4a1d-bef2-dbf6521a0c02 admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:05.147407 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf1f616-0794-4a1d-bef2-dbf6521a0c02 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:13:05.147713 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf1f616-0794-4a1d-bef2-dbf6521a0c02 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:05.149105 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c4a4f-5b62-47c4-adf3-a56432d7cc54 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:05.149328 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf1f616-0794-4a1d-bef2-dbf6521a0c02 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:05.153888 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da8607e-cf15-448b-9613-10aa6f55200d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:05.154547 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da8607e-cf15-448b-9613-10aa6f55200d admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:05.154830 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da8607e-cf15-448b-9613-10aa6f55200d admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:05.155108 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da8607e-cf15-448b-9613-10aa6f55200d admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:05.155868 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da8607e-cf15-448b-9613-10aa6f55200d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:05.158374 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da8607e-cf15-448b-9613-10aa6f55200d admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:05.161684 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 796/1567] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:13:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:13:05.162469 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf1f616-0794-4a1d-bef2-dbf6521a0c02 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:05.178488 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf1f616-0794-4a1d-bef2-dbf6521a0c02 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:05.183386 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937e15d-75a4-497c-a664-e773bd6f77f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:05.184076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937e15d-75a4-497c-a664-e773bd6f77f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:05.184370 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937e15d-75a4-497c-a664-e773bd6f77f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:05.184653 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937e15d-75a4-497c-a664-e773bd6f77f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:05.185427 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9937e15d-75a4-497c-a664-e773bd6f77f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:13:05.185738 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9937e15d-75a4-497c-a664-e773bd6f77f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:05.187384 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9937e15d-75a4-497c-a664-e773bd6f77f0 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:05.199326 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 797/1568] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:13:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:13:05.200142 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937e15d-75a4-497c-a664-e773bd6f77f0 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:05.209891 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937e15d-75a4-497c-a664-e773bd6f77f0 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:05.221723 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ab5ce-0a61-475f-aa58-26deb1147ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:05.221723 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ab5ce-0a61-475f-aa58-26deb1147ae4 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:05.221723 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ab5ce-0a61-475f-aa58-26deb1147ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:05.221723 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ab5ce-0a61-475f-aa58-26deb1147ae4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:05.222150 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33ab5ce-0a61-475f-aa58-26deb1147ae4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:13:05.222150 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33ab5ce-0a61-475f-aa58-26deb1147ae4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:05.222150 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33ab5ce-0a61-475f-aa58-26deb1147ae4 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:05.229931 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 798/1569] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:13:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:13:05.230848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ab5ce-0a61-475f-aa58-26deb1147ae4 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:05.240054 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ab5ce-0a61-475f-aa58-26deb1147ae4 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:05.245119 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cead5c0-39f3-49a3-853e-a4878ff5d845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:05.245755 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cead5c0-39f3-49a3-853e-a4878ff5d845 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:05.246034 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cead5c0-39f3-49a3-853e-a4878ff5d845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:05.246305 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cead5c0-39f3-49a3-853e-a4878ff5d845 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:05.247009 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cead5c0-39f3-49a3-853e-a4878ff5d845 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:05.248997 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cead5c0-39f3-49a3-853e-a4878ff5d845 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:05.256995 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 772/1570] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:13:05 2022] POST /identity/v3/projects => generated 381 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:05.257912 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da8607e-cf15-448b-9613-10aa6f55200d admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:05.317993 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da8607e-cf15-448b-9613-10aa6f55200d admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:05.322428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d7cdc9-bd78-4fe2-b5b6-a8afdb8cfa5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:05.323081 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d7cdc9-bd78-4fe2-b5b6-a8afdb8cfa5e admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:05.323391 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d7cdc9-bd78-4fe2-b5b6-a8afdb8cfa5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:05.323665 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d7cdc9-bd78-4fe2-b5b6-a8afdb8cfa5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:05.324410 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d7cdc9-bd78-4fe2-b5b6-a8afdb8cfa5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:05.326006 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d7cdc9-bd78-4fe2-b5b6-a8afdb8cfa5e admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:05.336128 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 773/1571] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:05 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:05.336916 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d7cdc9-bd78-4fe2-b5b6-a8afdb8cfa5e admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:05.343117 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 799/1572] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:13:05 2022] POST /identity/v3/projects => generated 391 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:05.343999 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cead5c0-39f3-49a3-853e-a4878ff5d845 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:05.349493 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d7cdc9-bd78-4fe2-b5b6-a8afdb8cfa5e admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:05.404971 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3239f-9b7d-4d37-a26b-f29f388f03bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:05.405638 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3239f-9b7d-4d37-a26b-f29f388f03bf admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:05.405922 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3239f-9b7d-4d37-a26b-f29f388f03bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:05.406197 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3239f-9b7d-4d37-a26b-f29f388f03bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:05.406981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa3239f-9b7d-4d37-a26b-f29f388f03bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:05.408534 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cead5c0-39f3-49a3-853e-a4878ff5d845 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:05.409119 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa3239f-9b7d-4d37-a26b-f29f388f03bf admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:05.413179 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329730bf-2ef3-4958-9f40-665243ddef42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:05.413846 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329730bf-2ef3-4958-9f40-665243ddef42 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:05.414134 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329730bf-2ef3-4958-9f40-665243ddef42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:05.414413 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329730bf-2ef3-4958-9f40-665243ddef42 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:05.415099 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329730bf-2ef3-4958-9f40-665243ddef42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:05.417031 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329730bf-2ef3-4958-9f40-665243ddef42 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:05.419471 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 774/1573] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:05 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:05.420347 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3239f-9b7d-4d37-a26b-f29f388f03bf admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:05.430711 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3239f-9b7d-4d37-a26b-f29f388f03bf admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:05.435261 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb56de-5b1a-4dc1-a2ad-7e448f0e5b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:05.440740 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb56de-5b1a-4dc1-a2ad-7e448f0e5b6c admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:05.440740 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb56de-5b1a-4dc1-a2ad-7e448f0e5b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:05.440740 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cb56de-5b1a-4dc1-a2ad-7e448f0e5b6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:05.440740 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cb56de-5b1a-4dc1-a2ad-7e448f0e5b6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:05.440740 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cb56de-5b1a-4dc1-a2ad-7e448f0e5b6c admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:05.529335 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 800/1574] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:13:05 2022] POST /identity/v3/users => generated 293 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:05.530404 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329730bf-2ef3-4958-9f40-665243ddef42 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:05.540254 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329730bf-2ef3-4958-9f40-665243ddef42 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:05.544940 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8885da8-165c-4408-8482-32b1794e0a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:05.545578 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8885da8-165c-4408-8482-32b1794e0a01 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:05.545867 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8885da8-165c-4408-8482-32b1794e0a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:05.546149 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8885da8-165c-4408-8482-32b1794e0a01 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:05.546856 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8885da8-165c-4408-8482-32b1794e0a01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:05.548843 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8885da8-165c-4408-8482-32b1794e0a01 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:05.557144 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:05.564537 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 775/1575] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:13:05 2022] POST /identity/v3/projects => generated 379 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:05.564537 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb56de-5b1a-4dc1-a2ad-7e448f0e5b6c admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:05.578431 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:05.619194 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cb56de-5b1a-4dc1-a2ad-7e448f0e5b6c admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:05.625258 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 801/1576] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:13:05 2022] POST /identity/v3/projects => generated 375 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:05.625258 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8885da8-165c-4408-8482-32b1794e0a01 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:05.628608 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39dd4c5-dd00-43d8-98ff-e45fc56ec42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:05.629634 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39dd4c5-dd00-43d8-98ff-e45fc56ec42b admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:05.630087 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39dd4c5-dd00-43d8-98ff-e45fc56ec42b admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:05.630542 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39dd4c5-dd00-43d8-98ff-e45fc56ec42b admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:05.631617 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39dd4c5-dd00-43d8-98ff-e45fc56ec42b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:05.634932 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39dd4c5-dd00-43d8-98ff-e45fc56ec42b admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:05.691549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8885da8-165c-4408-8482-32b1794e0a01 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:05.696265 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5411b54-5547-495b-9c15-18e075a339b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:05.696937 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5411b54-5547-495b-9c15-18e075a339b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:05.697680 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5411b54-5547-495b-9c15-18e075a339b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:05.697680 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5411b54-5547-495b-9c15-18e075a339b5 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:05.698291 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5411b54-5547-495b-9c15-18e075a339b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:05.700312 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5411b54-5547-495b-9c15-18e075a339b5 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:05.721485 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 776/1577] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:13:05 2022] POST /identity/v3/projects => generated 365 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:05.721485 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39dd4c5-dd00-43d8-98ff-e45fc56ec42b admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:05.783129 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39dd4c5-dd00-43d8-98ff-e45fc56ec42b admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:05.787906 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d72763c-901e-4a27-a1ac-4e532a43afa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:05.788572 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72763c-901e-4a27-a1ac-4e532a43afa8 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:05.788860 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72763c-901e-4a27-a1ac-4e532a43afa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:05.789139 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d72763c-901e-4a27-a1ac-4e532a43afa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:05.789864 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d72763c-901e-4a27-a1ac-4e532a43afa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:05.791461 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d72763c-901e-4a27-a1ac-4e532a43afa8 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:05.801295 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 777/1578] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:05 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:05.802078 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d72763c-901e-4a27-a1ac-4e532a43afa8 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:05.806689 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 802/1579] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:13:05 2022] POST /identity/v3/users => generated 291 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:05.806689 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5411b54-5547-495b-9c15-18e075a339b5 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:05.812141 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d72763c-901e-4a27-a1ac-4e532a43afa8 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:05.816571 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5411b54-5547-495b-9c15-18e075a339b5 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:05.817212 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9af9cd0-7048-40d8-816b-df252cbe2bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:05.817867 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af9cd0-7048-40d8-816b-df252cbe2bb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:05.818154 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af9cd0-7048-40d8-816b-df252cbe2bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:05.818437 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af9cd0-7048-40d8-816b-df252cbe2bb5 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:05.819106 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9af9cd0-7048-40d8-816b-df252cbe2bb5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:05.821033 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9af9cd0-7048-40d8-816b-df252cbe2bb5 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:05.821235 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225792df-76e9-4a78-883c-df7d096c4d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:05.822107 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225792df-76e9-4a78-883c-df7d096c4d6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:05.822387 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225792df-76e9-4a78-883c-df7d096c4d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:05.822662 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225792df-76e9-4a78-883c-df7d096c4d6a admin admin] PATH_INFO: `/v3/projects/e7912de4ef474c2b85310fd847a59bdd/users/f4dc9ce5f5eb48759bd5bf1fdf46d310/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:05.838824 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225792df-76e9-4a78-883c-df7d096c4d6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7912de4ef474c2b85310fd847a59bdd, user_id=f4dc9ce5f5eb48759bd5bf1fdf46d310, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:05.839866 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: WARNING py.warnings [None req-225792df-76e9-4a78-883c-df7d096c4d6a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 06 17:13:05.839866 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: warnings.warn(msg) Jul 06 17:13:05.839866 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: Jul 06 17:13:05.841764 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225792df-76e9-4a78-883c-df7d096c4d6a admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:05.883719 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 778/1580] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:13:05 2022] POST /identity/v3/users => generated 296 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:05.884658 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9af9cd0-7048-40d8-816b-df252cbe2bb5 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:05.894789 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9af9cd0-7048-40d8-816b-df252cbe2bb5 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:05.900527 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd4ada-e569-4e93-a63d-3391ba43c452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:05.906249 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd4ada-e569-4e93-a63d-3391ba43c452 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:05.906249 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd4ada-e569-4e93-a63d-3391ba43c452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:05.906249 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badd4ada-e569-4e93-a63d-3391ba43c452 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:05.906249 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badd4ada-e569-4e93-a63d-3391ba43c452 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:05.906249 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-badd4ada-e569-4e93-a63d-3391ba43c452 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:05.910864 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 803/1581] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:05 2022] PUT /identity/v3/projects/e7912de4ef474c2b85310fd847a59bdd/users/f4dc9ce5f5eb48759bd5bf1fdf46d310/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:05.911757 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225792df-76e9-4a78-883c-df7d096c4d6a admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:05.914877 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 779/1582] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:05.915756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd4ada-e569-4e93-a63d-3391ba43c452 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:05.925921 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-badd4ada-e569-4e93-a63d-3391ba43c452 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:05.976225 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225792df-76e9-4a78-883c-df7d096c4d6a admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:05.980866 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cce6d7-c4db-4a21-a253-bceabb29d9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:05.981615 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cce6d7-c4db-4a21-a253-bceabb29d9ac admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:05.981976 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cce6d7-c4db-4a21-a253-bceabb29d9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:05.982119 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c1b7c-9d77-450b-a8c3-f82daa9f4161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:05.982395 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cce6d7-c4db-4a21-a253-bceabb29d9ac admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:05.982817 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c1b7c-9d77-450b-a8c3-f82daa9f4161 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:05.983103 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c1b7c-9d77-450b-a8c3-f82daa9f4161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:05.983282 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cce6d7-c4db-4a21-a253-bceabb29d9ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:05.983379 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c1b7c-9d77-450b-a8c3-f82daa9f4161 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:05.984082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8c1b7c-9d77-450b-a8c3-f82daa9f4161 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:05.985934 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cce6d7-c4db-4a21-a253-bceabb29d9ac admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:05.986656 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8c1b7c-9d77-450b-a8c3-f82daa9f4161 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:06.060426 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 804/1583] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:13:05 2022] POST /identity/v3/users => generated 290 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:06.061214 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cce6d7-c4db-4a21-a253-bceabb29d9ac admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:06.067842 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:06.071785 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cce6d7-c4db-4a21-a253-bceabb29d9ac admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:06.072987 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 780/1584] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:13:05 2022] POST /identity/v3/users => generated 288 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:06.073726 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c1b7c-9d77-450b-a8c3-f82daa9f4161 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:06.076761 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202aa646-c734-4792-9fde-202729885c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:06.077421 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202aa646-c734-4792-9fde-202729885c7d admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:06.077706 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202aa646-c734-4792-9fde-202729885c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:06.077983 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202aa646-c734-4792-9fde-202729885c7d admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:06.078802 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202aa646-c734-4792-9fde-202729885c7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:06.080774 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202aa646-c734-4792-9fde-202729885c7d admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:06.083049 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c1b7c-9d77-450b-a8c3-f82daa9f4161 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:06.087904 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e471f92b-26f6-4fc9-bec1-063f10eacaa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:06.088772 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471f92b-26f6-4fc9-bec1-063f10eacaa0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:06.089109 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471f92b-26f6-4fc9-bec1-063f10eacaa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:06.089552 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e471f92b-26f6-4fc9-bec1-063f10eacaa0 admin admin] PATH_INFO: `/v3/projects/0f928ce25a9d4c419769458770d9c7cc/users/4012c27d84474dc1930ace9990b2bd8b/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:06.103709 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e471f92b-26f6-4fc9-bec1-063f10eacaa0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f928ce25a9d4c419769458770d9c7cc, user_id=4012c27d84474dc1930ace9990b2bd8b, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:06.106406 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e471f92b-26f6-4fc9-bec1-063f10eacaa0 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:06.158200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 805/1585] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:13:06 2022] POST /identity/v3/users => generated 283 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:06.159096 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202aa646-c734-4792-9fde-202729885c7d admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:06.168514 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202aa646-c734-4792-9fde-202729885c7d admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:06.173510 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d123e824-7591-4a45-9143-46d1b5284ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:06.174203 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123e824-7591-4a45-9143-46d1b5284ebb admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:06.174508 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123e824-7591-4a45-9143-46d1b5284ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:06.174784 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d123e824-7591-4a45-9143-46d1b5284ebb admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:06.175518 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d123e824-7591-4a45-9143-46d1b5284ebb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:06.177120 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d123e824-7591-4a45-9143-46d1b5284ebb admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:06.187150 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 781/1586] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:06 2022] PUT /identity/v3/projects/0f928ce25a9d4c419769458770d9c7cc/users/4012c27d84474dc1930ace9990b2bd8b/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:06.190025 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e471f92b-26f6-4fc9-bec1-063f10eacaa0 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:06.190148 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 806/1587] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:06.190812 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687ddca-cfa6-4900-97f3-9154f4dd404c None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:06.191101 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687ddca-cfa6-4900-97f3-9154f4dd404c None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:06.191377 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687ddca-cfa6-4900-97f3-9154f4dd404c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:06.217901 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-a687ddca-cfa6-4900-97f3-9154f4dd404c None None] MFA Rules not processed for user `f4dc9ce5f5eb48759bd5bf1fdf46d310`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:06.224716 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-a687ddca-cfa6-4900-97f3-9154f4dd404c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:06.224716 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:06.253571 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e471f92b-26f6-4fc9-bec1-063f10eacaa0 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:06.258298 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecc5852-e32e-40c5-af75-cb1d1c24709e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:06.258990 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc5852-e32e-40c5-af75-cb1d1c24709e admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:06.259398 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc5852-e32e-40c5-af75-cb1d1c24709e admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:06.259679 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecc5852-e32e-40c5-af75-cb1d1c24709e admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:06.260447 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecc5852-e32e-40c5-af75-cb1d1c24709e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:06.262085 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecc5852-e32e-40c5-af75-cb1d1c24709e admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:06.272732 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 782/1588] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:06.273535 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecc5852-e32e-40c5-af75-cb1d1c24709e admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:06.283182 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecc5852-e32e-40c5-af75-cb1d1c24709e admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:06.293239 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 807/1589] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:06 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:06.293382 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c8025-1080-4b61-b369-fc9dfce5ec36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:06.293522 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687ddca-cfa6-4900-97f3-9154f4dd404c None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:06.293707 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c8025-1080-4b61-b369-fc9dfce5ec36 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:06.293707 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c8025-1080-4b61-b369-fc9dfce5ec36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:06.293707 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c8025-1080-4b61-b369-fc9dfce5ec36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:06.293707 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780c8025-1080-4b61-b369-fc9dfce5ec36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:06.293707 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780c8025-1080-4b61-b369-fc9dfce5ec36 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:06.299998 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687ddca-cfa6-4900-97f3-9154f4dd404c None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:06.302631 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 783/1590] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:06.303471 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c8025-1080-4b61-b369-fc9dfce5ec36 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:06.305490 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05d4473-9d05-4d6d-ae09-ac5ee980daa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:06.306150 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d4473-9d05-4d6d-ae09-ac5ee980daa3 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:06.306696 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d4473-9d05-4d6d-ae09-ac5ee980daa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:06.307211 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d4473-9d05-4d6d-ae09-ac5ee980daa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:06.308480 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05d4473-9d05-4d6d-ae09-ac5ee980daa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:06.310127 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05d4473-9d05-4d6d-ae09-ac5ee980daa3 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:06.312868 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c8025-1080-4b61-b369-fc9dfce5ec36 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:06.317877 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee613565-97b2-4b2b-bf0d-523cd73c77ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:06.318511 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee613565-97b2-4b2b-bf0d-523cd73c77ac admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:06.318796 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee613565-97b2-4b2b-bf0d-523cd73c77ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:06.319074 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee613565-97b2-4b2b-bf0d-523cd73c77ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:06.319815 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee613565-97b2-4b2b-bf0d-523cd73c77ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:06.320736 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 808/1591] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:06.321383 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee613565-97b2-4b2b-bf0d-523cd73c77ac admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:06.321498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05d4473-9d05-4d6d-ae09-ac5ee980daa3 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:06.330956 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05d4473-9d05-4d6d-ae09-ac5ee980daa3 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:06.331754 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 784/1592] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:06.332534 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee613565-97b2-4b2b-bf0d-523cd73c77ac admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:06.337768 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7696320d-ba7b-413d-946e-d9d50f9f049d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:06.338412 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7696320d-ba7b-413d-946e-d9d50f9f049d admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:06.338694 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7696320d-ba7b-413d-946e-d9d50f9f049d admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:06.338971 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7696320d-ba7b-413d-946e-d9d50f9f049d admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:06.339774 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7696320d-ba7b-413d-946e-d9d50f9f049d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:06.341407 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7696320d-ba7b-413d-946e-d9d50f9f049d admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:06.342066 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee613565-97b2-4b2b-bf0d-523cd73c77ac admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:06.347112 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6348c335-3947-421e-bc7e-9fa53531524d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:06.347785 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6348c335-3947-421e-bc7e-9fa53531524d admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:06.348080 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6348c335-3947-421e-bc7e-9fa53531524d admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:06.348359 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6348c335-3947-421e-bc7e-9fa53531524d admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:06.349077 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6348c335-3947-421e-bc7e-9fa53531524d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:06.350625 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6348c335-3947-421e-bc7e-9fa53531524d admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:06.351801 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 809/1593] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:06.353001 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3292b0-d42c-4639-83f7-8a6535413117 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:06.353292 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3292b0-d42c-4639-83f7-8a6535413117 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:06.353596 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3292b0-d42c-4639-83f7-8a6535413117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:06.366041 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 785/1594] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:06.366041 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6348c335-3947-421e-bc7e-9fa53531524d admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:06.373678 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6348c335-3947-421e-bc7e-9fa53531524d admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:06.378363 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c68f46-f81d-457b-bc29-33ca1ee7ec77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:06.378990 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c68f46-f81d-457b-bc29-33ca1ee7ec77 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:06.379274 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c68f46-f81d-457b-bc29-33ca1ee7ec77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:06.379550 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c68f46-f81d-457b-bc29-33ca1ee7ec77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:06.379957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-ab3292b0-d42c-4639-83f7-8a6535413117 None None] MFA Rules not processed for user `4012c27d84474dc1930ace9990b2bd8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:06.380345 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c68f46-f81d-457b-bc29-33ca1ee7ec77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:06.381154 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-ab3292b0-d42c-4639-83f7-8a6535413117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:06.381925 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c68f46-f81d-457b-bc29-33ca1ee7ec77 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:06.391771 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 786/1595] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:06.392471 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c68f46-f81d-457b-bc29-33ca1ee7ec77 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:06.402719 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c68f46-f81d-457b-bc29-33ca1ee7ec77 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:06.407756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322e6475-f076-4338-a7db-ae5ddc18f9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:06.408393 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e6475-f076-4338-a7db-ae5ddc18f9c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:06.408705 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e6475-f076-4338-a7db-ae5ddc18f9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:06.408978 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322e6475-f076-4338-a7db-ae5ddc18f9c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:06.409675 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322e6475-f076-4338-a7db-ae5ddc18f9c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:06.411526 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322e6475-f076-4338-a7db-ae5ddc18f9c3 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:06.445543 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 810/1596] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:06 2022] POST /identity/v3/auth/tokens => generated 3220 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:06.446353 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3292b0-d42c-4639-83f7-8a6535413117 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:06.456159 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3292b0-d42c-4639-83f7-8a6535413117 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:06.460910 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5653be3-1ea7-4ae4-8ece-9b64c4e6ec75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:06.461753 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5653be3-1ea7-4ae4-8ece-9b64c4e6ec75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:06.462035 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5653be3-1ea7-4ae4-8ece-9b64c4e6ec75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:06.462307 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5653be3-1ea7-4ae4-8ece-9b64c4e6ec75 admin admin] PATH_INFO: `/v3/projects/71698505c2fa4176a7dbcb62ef137649/users/5ace6486a5d942e08f3f4708a4193e4a/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:06.476918 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5653be3-1ea7-4ae4-8ece-9b64c4e6ec75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71698505c2fa4176a7dbcb62ef137649, user_id=5ace6486a5d942e08f3f4708a4193e4a, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:06.479524 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5653be3-1ea7-4ae4-8ece-9b64c4e6ec75 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:06.502703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 787/1597] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:13:06 2022] POST /identity/v3/projects => generated 405 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:06.503532 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322e6475-f076-4338-a7db-ae5ddc18f9c3 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:06.562651 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322e6475-f076-4338-a7db-ae5ddc18f9c3 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:06.567184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecfc837-3625-49b1-b19d-7252df2a5edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:06.567863 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecfc837-3625-49b1-b19d-7252df2a5edf admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:06.570843 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecfc837-3625-49b1-b19d-7252df2a5edf admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:06.570843 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecfc837-3625-49b1-b19d-7252df2a5edf admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:06.570843 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecfc837-3625-49b1-b19d-7252df2a5edf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:06.570843 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecfc837-3625-49b1-b19d-7252df2a5edf admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:06.580681 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 788/1598] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:06.581557 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecfc837-3625-49b1-b19d-7252df2a5edf admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:06.586915 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 811/1599] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:06 2022] PUT /identity/v3/projects/71698505c2fa4176a7dbcb62ef137649/users/5ace6486a5d942e08f3f4708a4193e4a/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:06.587639 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5653be3-1ea7-4ae4-8ece-9b64c4e6ec75 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:06.588277 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:06.597554 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecfc837-3625-49b1-b19d-7252df2a5edf admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:06.637208 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:06.650220 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ba6f59-b530-4b59-8f50-f8911d33f5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:06.650636 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5653be3-1ea7-4ae4-8ece-9b64c4e6ec75 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:06.650890 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba6f59-b530-4b59-8f50-f8911d33f5f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:06.651178 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba6f59-b530-4b59-8f50-f8911d33f5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:06.651719 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ba6f59-b530-4b59-8f50-f8911d33f5f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:06.652473 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ba6f59-b530-4b59-8f50-f8911d33f5f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:06.654071 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ba6f59-b530-4b59-8f50-f8911d33f5f4 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:06.655835 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c2c56-6604-44c8-8d52-bcb91dbd1dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:06.657435 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c2c56-6604-44c8-8d52-bcb91dbd1dfe admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:06.657618 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c2c56-6604-44c8-8d52-bcb91dbd1dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:06.657897 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c2c56-6604-44c8-8d52-bcb91dbd1dfe admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:06.658617 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434c2c56-6604-44c8-8d52-bcb91dbd1dfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:06.660680 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434c2c56-6604-44c8-8d52-bcb91dbd1dfe admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:06.664319 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 789/1600] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:06.665282 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ba6f59-b530-4b59-8f50-f8911d33f5f4 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:06.670910 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 812/1601] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:06.671555 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c2c56-6604-44c8-8d52-bcb91dbd1dfe admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:06.678221 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ba6f59-b530-4b59-8f50-f8911d33f5f4 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:06.683090 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8df8d8-c72d-410e-bdf7-17a55e934c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:06.683886 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c2c56-6604-44c8-8d52-bcb91dbd1dfe admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:06.684020 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8df8d8-c72d-410e-bdf7-17a55e934c96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:06.684268 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8df8d8-c72d-410e-bdf7-17a55e934c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:06.684545 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8df8d8-c72d-410e-bdf7-17a55e934c96 admin admin] PATH_INFO: `/v3/projects/0ee8f6d39e1d43f297be3cb8e9e5461c/users/79ef7dc2d5d247b68ab8a9395faa0e40/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:06.688580 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5940bd-16a8-4a13-a2e1-fb13daa1f5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:06.689438 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5940bd-16a8-4a13-a2e1-fb13daa1f5a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:06.689719 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5940bd-16a8-4a13-a2e1-fb13daa1f5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:06.689993 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5940bd-16a8-4a13-a2e1-fb13daa1f5a9 admin admin] PATH_INFO: `/v3/projects/13a0db3fcd71427e9a442c589f435eb4/users/8e302cf8d56c4db8bb8480ec234ac908/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:06.699249 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8df8d8-c72d-410e-bdf7-17a55e934c96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ee8f6d39e1d43f297be3cb8e9e5461c, user_id=79ef7dc2d5d247b68ab8a9395faa0e40, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:06.701908 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8df8d8-c72d-410e-bdf7-17a55e934c96 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:06.706726 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5940bd-16a8-4a13-a2e1-fb13daa1f5a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=13a0db3fcd71427e9a442c589f435eb4, user_id=8e302cf8d56c4db8bb8480ec234ac908, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:06.709405 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5940bd-16a8-4a13-a2e1-fb13daa1f5a9 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:06.763841 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 790/1602] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:06 2022] PUT /identity/v3/projects/0ee8f6d39e1d43f297be3cb8e9e5461c/users/79ef7dc2d5d247b68ab8a9395faa0e40/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:06.764729 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8df8d8-c72d-410e-bdf7-17a55e934c96 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:06.775372 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 813/1603] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:06 2022] PUT /identity/v3/projects/13a0db3fcd71427e9a442c589f435eb4/users/8e302cf8d56c4db8bb8480ec234ac908/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:06.776598 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5940bd-16a8-4a13-a2e1-fb13daa1f5a9 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:06.832895 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8df8d8-c72d-410e-bdf7-17a55e934c96 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:06.837525 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cccc6ea-b958-4d39-917a-907d56b698b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:06.837971 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5940bd-16a8-4a13-a2e1-fb13daa1f5a9 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:06.838186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cccc6ea-b958-4d39-917a-907d56b698b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:06.838476 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cccc6ea-b958-4d39-917a-907d56b698b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:06.838756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cccc6ea-b958-4d39-917a-907d56b698b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:06.839453 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cccc6ea-b958-4d39-917a-907d56b698b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:06.841385 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cccc6ea-b958-4d39-917a-907d56b698b2 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:06.842901 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248bb22d-b192-4997-8499-0f755967646d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:06.843568 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248bb22d-b192-4997-8499-0f755967646d admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:06.843888 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248bb22d-b192-4997-8499-0f755967646d admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:06.844169 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248bb22d-b192-4997-8499-0f755967646d admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:06.845435 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248bb22d-b192-4997-8499-0f755967646d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:06.847353 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248bb22d-b192-4997-8499-0f755967646d admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:06.947602 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 791/1604] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:13:06 2022] POST /identity/v3/projects => generated 355 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:06.948469 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cccc6ea-b958-4d39-917a-907d56b698b2 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:06.978795 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 814/1605] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:13:06 2022] POST /identity/v3/users => generated 309 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:06.978795 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b768dd-748d-4d97-96a7-e073067fc13a None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:06.978795 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b768dd-748d-4d97-96a7-e073067fc13a None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:06.978795 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b768dd-748d-4d97-96a7-e073067fc13a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:07.003591 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-70b768dd-748d-4d97-96a7-e073067fc13a None None] MFA Rules not processed for user `5ace6486a5d942e08f3f4708a4193e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:07.005062 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-70b768dd-748d-4d97-96a7-e073067fc13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:07.009072 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cccc6ea-b958-4d39-917a-907d56b698b2 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:07.013740 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd7a6fa-ca88-40da-af51-a14ba59ba4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:07.014623 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd7a6fa-ca88-40da-af51-a14ba59ba4f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:07.014911 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd7a6fa-ca88-40da-af51-a14ba59ba4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:07.015189 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd7a6fa-ca88-40da-af51-a14ba59ba4f8 admin admin] PATH_INFO: `/v3/projects/ae7e64bccf304d5ebed6c23c90d47d7a/users/8e72875df16d497f9a542da0a5636f5f/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:07.018038 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:07.030829 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd7a6fa-ca88-40da-af51-a14ba59ba4f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae7e64bccf304d5ebed6c23c90d47d7a, user_id=8e72875df16d497f9a542da0a5636f5f, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:07.034331 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd7a6fa-ca88-40da-af51-a14ba59ba4f8 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:07.069894 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 815/1606] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:07.070696 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b768dd-748d-4d97-96a7-e073067fc13a None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:07.082378 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b768dd-748d-4d97-96a7-e073067fc13a None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:07.086965 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04be36-a0b9-4454-8a7f-1f954e802032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:07.087914 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04be36-a0b9-4454-8a7f-1f954e802032 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:07.088198 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04be36-a0b9-4454-8a7f-1f954e802032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:07.088471 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04be36-a0b9-4454-8a7f-1f954e802032 admin admin] PATH_INFO: `/v3/projects/f3d0d78164dc48e5aa8dc393c9071f4a/users/7922db6ad0ce41868a68f5a834a3b8f3/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:07.104354 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca04be36-a0b9-4454-8a7f-1f954e802032 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3d0d78164dc48e5aa8dc393c9071f4a, user_id=7922db6ad0ce41868a68f5a834a3b8f3, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:07.107110 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca04be36-a0b9-4454-8a7f-1f954e802032 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:07.113922 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 792/1607] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:06 2022] PUT /identity/v3/projects/ae7e64bccf304d5ebed6c23c90d47d7a/users/8e72875df16d497f9a542da0a5636f5f/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:07.113922 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd7a6fa-ca88-40da-af51-a14ba59ba4f8 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:07.172729 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 816/1608] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:07 2022] PUT /identity/v3/projects/f3d0d78164dc48e5aa8dc393c9071f4a/users/7922db6ad0ce41868a68f5a834a3b8f3/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:07.172729 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793cf035-b402-4655-bb9c-bdbf7d0d0591 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:07.172729 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793cf035-b402-4655-bb9c-bdbf7d0d0591 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:07.172729 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793cf035-b402-4655-bb9c-bdbf7d0d0591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:07.175959 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd7a6fa-ca88-40da-af51-a14ba59ba4f8 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:07.199455 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-793cf035-b402-4655-bb9c-bdbf7d0d0591 None None] MFA Rules not processed for user `79ef7dc2d5d247b68ab8a9395faa0e40`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:07.201087 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-793cf035-b402-4655-bb9c-bdbf7d0d0591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:07.205674 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:07.232113 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08607d4a-534a-4970-a1ab-3513bf9734d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:07.233014 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08607d4a-534a-4970-a1ab-3513bf9734d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:07.233301 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08607d4a-534a-4970-a1ab-3513bf9734d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:07.233579 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08607d4a-534a-4970-a1ab-3513bf9734d8 admin admin] PATH_INFO: `/v3/projects/42690acd6bce45ad9690e99f29f4552e/users/cc18962150634d0ead8afdb9c3f85074/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:07.248524 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08607d4a-534a-4970-a1ab-3513bf9734d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42690acd6bce45ad9690e99f29f4552e, user_id=cc18962150634d0ead8afdb9c3f85074, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:07.251689 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08607d4a-534a-4970-a1ab-3513bf9734d8 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:07.269164 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 817/1609] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:07.270461 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439dff0-b311-40f1-a6c7-b9cf57ab1d6a None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:07.270998 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439dff0-b311-40f1-a6c7-b9cf57ab1d6a None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:07.271476 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6439dff0-b311-40f1-a6c7-b9cf57ab1d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:07.288955 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-6439dff0-b311-40f1-a6c7-b9cf57ab1d6a None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:07.290150 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-6439dff0-b311-40f1-a6c7-b9cf57ab1d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:07.295048 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 793/1610] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:07 2022] PUT /identity/v3/projects/42690acd6bce45ad9690e99f29f4552e/users/cc18962150634d0ead8afdb9c3f85074/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:07.295946 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08607d4a-534a-4970-a1ab-3513bf9734d8 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:07.357800 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08607d4a-534a-4970-a1ab-3513bf9734d8 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:07.358241 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 818/1611] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:07.359277 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6439dff0-b311-40f1-a6c7-b9cf57ab1d6a None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:07.364755 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ec5fee-a191-4e1f-b1b7-06ad53955223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:07.365541 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec5fee-a191-4e1f-b1b7-06ad53955223 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:07.365924 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec5fee-a191-4e1f-b1b7-06ad53955223 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:07.366296 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec5fee-a191-4e1f-b1b7-06ad53955223 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:07.367082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ec5fee-a191-4e1f-b1b7-06ad53955223 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:07.369034 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6439dff0-b311-40f1-a6c7-b9cf57ab1d6a None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:07.369280 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ec5fee-a191-4e1f-b1b7-06ad53955223 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:07.373605 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99696217-6558-461d-872d-a4a4111333f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:07.374259 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99696217-6558-461d-872d-a4a4111333f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:07.374546 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99696217-6558-461d-872d-a4a4111333f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:07.374822 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99696217-6558-461d-872d-a4a4111333f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:07.376107 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99696217-6558-461d-872d-a4a4111333f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:07.377687 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99696217-6558-461d-872d-a4a4111333f4 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:07.387506 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 819/1612] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:07.388906 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffacf09-dda8-40eb-9448-096fd01fc085 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:07.389203 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffacf09-dda8-40eb-9448-096fd01fc085 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:07.389480 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffacf09-dda8-40eb-9448-096fd01fc085 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:07.391039 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 820/1613] 10.222.0.40 () {54 vars in 961 bytes} [Wed Jul 6 17:13:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:13:07.392710 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863908b-bc44-4a26-b801-e96b1936ebb9 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:07.393065 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863908b-bc44-4a26-b801-e96b1936ebb9 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:07.393350 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863908b-bc44-4a26-b801-e96b1936ebb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:07.410666 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-7863908b-bc44-4a26-b801-e96b1936ebb9 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:07.411879 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-7863908b-bc44-4a26-b801-e96b1936ebb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:07.419349 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 821/1614] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:07.425019 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f66ca-df09-495b-8a8a-f5d74bdaec8a None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:07.425019 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f66ca-df09-495b-8a8a-f5d74bdaec8a None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:07.425019 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232f66ca-df09-495b-8a8a-f5d74bdaec8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:07.437326 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-232f66ca-df09-495b-8a8a-f5d74bdaec8a None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:07.439098 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-232f66ca-df09-495b-8a8a-f5d74bdaec8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:07.445142 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 794/1615] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:13:07 2022] POST /identity/v3/users => generated 284 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:07.446112 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ec5fee-a191-4e1f-b1b7-06ad53955223 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:07.449395 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 822/1616] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:07.455812 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5832d3b8-3188-4778-b80d-69c7b66d10d7 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:07.455812 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5832d3b8-3188-4778-b80d-69c7b66d10d7 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:07.455812 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5832d3b8-3188-4778-b80d-69c7b66d10d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:07.456150 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:07.459202 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ec5fee-a191-4e1f-b1b7-06ad53955223 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:07.463595 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ddc08f-409f-4fbd-9bff-a3326521abfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:07.464399 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ddc08f-409f-4fbd-9bff-a3326521abfc admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:07.464787 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ddc08f-409f-4fbd-9bff-a3326521abfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:07.465215 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ddc08f-409f-4fbd-9bff-a3326521abfc admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:07.465951 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ddc08f-409f-4fbd-9bff-a3326521abfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:07.467956 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ddc08f-409f-4fbd-9bff-a3326521abfc admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:07.479077 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-5832d3b8-3188-4778-b80d-69c7b66d10d7 None None] MFA Rules not processed for user `cc18962150634d0ead8afdb9c3f85074`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:07.480401 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-5832d3b8-3188-4778-b80d-69c7b66d10d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:07.543896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 823/1617] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:07 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:07.544712 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5832d3b8-3188-4778-b80d-69c7b66d10d7 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:07.545971 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-5832d3b8-3188-4778-b80d-69c7b66d10d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:07.559056 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:07.573210 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 795/1618] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:13:07 2022] POST /identity/v3/projects => generated 381 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:07.573210 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ddc08f-409f-4fbd-9bff-a3326521abfc admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:07.603760 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5832d3b8-3188-4778-b80d-69c7b66d10d7 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:07.636213 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ddc08f-409f-4fbd-9bff-a3326521abfc admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:07.640800 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e4de8f-db60-4ed6-b9d9-394e6f1748dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:07.641687 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e4de8f-db60-4ed6-b9d9-394e6f1748dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:07.641973 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e4de8f-db60-4ed6-b9d9-394e6f1748dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:07.642251 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e4de8f-db60-4ed6-b9d9-394e6f1748dd admin admin] PATH_INFO: `/v3/projects/88ebcf54fab546aaa4645834ade22b8a/users/6ced420a85f84a56919ee517c40245a4/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:07.655058 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb477d-9b4e-4c64-85b3-c484970daea1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:07.655743 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb477d-9b4e-4c64-85b3-c484970daea1 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:07.656028 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb477d-9b4e-4c64-85b3-c484970daea1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:07.656314 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb477d-9b4e-4c64-85b3-c484970daea1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:07.664071 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e4de8f-db60-4ed6-b9d9-394e6f1748dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=88ebcf54fab546aaa4645834ade22b8a, user_id=6ced420a85f84a56919ee517c40245a4, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:07.665716 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fb477d-9b4e-4c64-85b3-c484970daea1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:07.666857 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e4de8f-db60-4ed6-b9d9-394e6f1748dd admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:07.667145 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fb477d-9b4e-4c64-85b3-c484970daea1 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:07.677037 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 824/1619] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:13:07.678257 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4680b96c-1df3-4404-a4ed-dbe4e050968e None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:07.682804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4680b96c-1df3-4404-a4ed-dbe4e050968e None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:07.682804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4680b96c-1df3-4404-a4ed-dbe4e050968e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:07.697890 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-4680b96c-1df3-4404-a4ed-dbe4e050968e None None] MFA Rules not processed for user `af26ef96dc9d47dc91549b736939268e`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:07.699101 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-4680b96c-1df3-4404-a4ed-dbe4e050968e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:07.764766 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 825/1620] 10.222.0.40 () {58 vars in 1058 bytes} [Wed Jul 6 17:13:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:07.765599 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4680b96c-1df3-4404-a4ed-dbe4e050968e None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:07.770643 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 796/1621] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:07 2022] PUT /identity/v3/projects/88ebcf54fab546aaa4645834ade22b8a/users/6ced420a85f84a56919ee517c40245a4/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:07.771596 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e4de8f-db60-4ed6-b9d9-394e6f1748dd admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:07.774563 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4680b96c-1df3-4404-a4ed-dbe4e050968e None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:07.828444 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f5331-09a6-41bc-9ccc-a0729b9c479b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:07.829264 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f5331-09a6-41bc-9ccc-a0729b9c479b service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:07.829700 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f5331-09a6-41bc-9ccc-a0729b9c479b service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:07.829988 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4f5331-09a6-41bc-9ccc-a0729b9c479b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:07.840434 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e4de8f-db60-4ed6-b9d9-394e6f1748dd admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:07.845682 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:07.847910 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9ce79-217d-4549-a324-c0b3518286b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:07.850602 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9ce79-217d-4549-a324-c0b3518286b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:07.850602 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9ce79-217d-4549-a324-c0b3518286b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:07.850602 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9ce79-217d-4549-a324-c0b3518286b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:07.850602 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d9ce79-217d-4549-a324-c0b3518286b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:07.851753 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d9ce79-217d-4549-a324-c0b3518286b4 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:07.863780 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 797/1622] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:07.867718 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9ce79-217d-4549-a324-c0b3518286b4 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:07.877732 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9ce79-217d-4549-a324-c0b3518286b4 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:07.882541 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6042d5b-b895-40fc-9ba2-83a9d70ecf35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:07.884990 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6042d5b-b895-40fc-9ba2-83a9d70ecf35 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:07.884990 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6042d5b-b895-40fc-9ba2-83a9d70ecf35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:07.884990 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6042d5b-b895-40fc-9ba2-83a9d70ecf35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:07.893485 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6042d5b-b895-40fc-9ba2-83a9d70ecf35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:07.895476 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6042d5b-b895-40fc-9ba2-83a9d70ecf35 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:07.896562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4f5331-09a6-41bc-9ccc-a0729b9c479b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:07.898005 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4f5331-09a6-41bc-9ccc-a0729b9c479b service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:07.908671 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 826/1623] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:13:07.910036 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 798/1624] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:13:07.910297 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f5331-09a6-41bc-9ccc-a0729b9c479b service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:07.910953 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6042d5b-b895-40fc-9ba2-83a9d70ecf35 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:07.925379 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6042d5b-b895-40fc-9ba2-83a9d70ecf35 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:07.930749 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9981a58e-abb3-4cb4-a3b1-8ecf1c78602a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:07.931602 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9981a58e-abb3-4cb4-a3b1-8ecf1c78602a admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:07.931780 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9981a58e-abb3-4cb4-a3b1-8ecf1c78602a admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:07.932061 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9981a58e-abb3-4cb4-a3b1-8ecf1c78602a admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:07.932743 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9981a58e-abb3-4cb4-a3b1-8ecf1c78602a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:07.934737 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9981a58e-abb3-4cb4-a3b1-8ecf1c78602a admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:07.971988 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4f5331-09a6-41bc-9ccc-a0729b9c479b service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:07.975753 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc48d13-80fb-4a8b-9886-394a2672f578 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:07.976411 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc48d13-80fb-4a8b-9886-394a2672f578 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:07.976718 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc48d13-80fb-4a8b-9886-394a2672f578 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:07.977092 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc48d13-80fb-4a8b-9886-394a2672f578 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:08.004255 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 799/1625] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:13:07 2022] POST /identity/v3/users => generated 297 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:08.005105 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9981a58e-abb3-4cb4-a3b1-8ecf1c78602a admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:08.044301 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:08.048518 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc48d13-80fb-4a8b-9886-394a2672f578 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:08.051570 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc48d13-80fb-4a8b-9886-394a2672f578 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:08.062430 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 827/1626] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:07 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:13:08.062951 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc48d13-80fb-4a8b-9886-394a2672f578 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:08.079188 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc48d13-80fb-4a8b-9886-394a2672f578 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:08.081023 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc1cbfd-435d-4456-bab9-0ec198d6d2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:08.081712 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc1cbfd-435d-4456-bab9-0ec198d6d2ac admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:08.082001 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc1cbfd-435d-4456-bab9-0ec198d6d2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:08.082602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc1cbfd-435d-4456-bab9-0ec198d6d2ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:08.083622 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc1cbfd-435d-4456-bab9-0ec198d6d2ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:08.085350 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc1cbfd-435d-4456-bab9-0ec198d6d2ac admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:08.100069 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 828/1627] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:08.100910 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc1cbfd-435d-4456-bab9-0ec198d6d2ac admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:08.109088 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9981a58e-abb3-4cb4-a3b1-8ecf1c78602a admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:08.114990 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc1cbfd-435d-4456-bab9-0ec198d6d2ac admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:08.117226 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3b8e07-2b6d-4fe0-b729-c2f9bd3cdf2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:08.117978 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b8e07-2b6d-4fe0-b729-c2f9bd3cdf2f service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:08.118372 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b8e07-2b6d-4fe0-b729-c2f9bd3cdf2f service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:08.118800 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3b8e07-2b6d-4fe0-b729-c2f9bd3cdf2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:08.120625 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c153c-a34e-4608-a0e6-347e75041d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:08.121597 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c153c-a34e-4608-a0e6-347e75041d9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:08.121942 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c153c-a34e-4608-a0e6-347e75041d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:08.122411 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9c153c-a34e-4608-a0e6-347e75041d9c admin admin] PATH_INFO: `/v3/projects/a8ace4333a914c72a33cbdfd5c8d6835/users/e8d2015ff7144a08aecfd1a2668d3809/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:08.140957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c153c-a34e-4608-a0e6-347e75041d9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8ace4333a914c72a33cbdfd5c8d6835, user_id=e8d2015ff7144a08aecfd1a2668d3809, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:08.143813 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9c153c-a34e-4608-a0e6-347e75041d9c admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:08.202580 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3b8e07-2b6d-4fe0-b729-c2f9bd3cdf2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:08.205245 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3b8e07-2b6d-4fe0-b729-c2f9bd3cdf2f service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:08.205747 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 829/1628] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:08 2022] PUT /identity/v3/projects/a8ace4333a914c72a33cbdfd5c8d6835/users/e8d2015ff7144a08aecfd1a2668d3809/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:08.213037 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c153c-a34e-4608-a0e6-347e75041d9c admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:08.234372 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 800/1629] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:13:08.235343 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3b8e07-2b6d-4fe0-b729-c2f9bd3cdf2f service nova] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:08.276092 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3b8e07-2b6d-4fe0-b729-c2f9bd3cdf2f service nova] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:08.293775 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:08.350262 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9c153c-a34e-4608-a0e6-347e75041d9c admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:08.355247 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b18610-78ed-4425-a6f3-95ce57071456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:08.356000 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b18610-78ed-4425-a6f3-95ce57071456 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:08.356345 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b18610-78ed-4425-a6f3-95ce57071456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:08.356673 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b18610-78ed-4425-a6f3-95ce57071456 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:08.358864 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b18610-78ed-4425-a6f3-95ce57071456 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:08.361508 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b18610-78ed-4425-a6f3-95ce57071456 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:08.378429 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 830/1630] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:08.378429 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b18610-78ed-4425-a6f3-95ce57071456 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:08.384943 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af2ddf-497d-4800-96a8-e059eabea59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:08.386674 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af2ddf-497d-4800-96a8-e059eabea59f admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:08.387162 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af2ddf-497d-4800-96a8-e059eabea59f admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:08.387601 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1af2ddf-497d-4800-96a8-e059eabea59f admin admin] PATH_INFO: `/v3/projects/88ebcf54fab546aaa4645834ade22b8a/users/6ced420a85f84a56919ee517c40245a4/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:08.388832 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b18610-78ed-4425-a6f3-95ce57071456 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:08.390007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1af2ddf-497d-4800-96a8-e059eabea59f admin admin] RBAC: Authorizing `identity:create_grant(project_id=88ebcf54fab546aaa4645834ade22b8a, user_id=6ced420a85f84a56919ee517c40245a4, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:08.396170 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2879ad4e-45c7-4a32-bbb5-719ebd518419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:08.397019 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879ad4e-45c7-4a32-bbb5-719ebd518419 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:08.402644 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879ad4e-45c7-4a32-bbb5-719ebd518419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:08.402644 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879ad4e-45c7-4a32-bbb5-719ebd518419 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:08.402644 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2879ad4e-45c7-4a32-bbb5-719ebd518419 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:08.402644 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2879ad4e-45c7-4a32-bbb5-719ebd518419 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:08.411905 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1af2ddf-497d-4800-96a8-e059eabea59f admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:08.420739 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 831/1631] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:08.420739 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2879ad4e-45c7-4a32-bbb5-719ebd518419 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:08.438775 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 801/1632] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:08 2022] PUT /identity/v3/projects/88ebcf54fab546aaa4645834ade22b8a/users/6ced420a85f84a56919ee517c40245a4/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:08.441742 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af2ddf-497d-4800-96a8-e059eabea59f admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:08.443919 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2879ad4e-45c7-4a32-bbb5-719ebd518419 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:08.535696 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dea7d6-88c2-444d-bc8f-f010ee53a1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:08.536910 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dea7d6-88c2-444d-bc8f-f010ee53a1eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:08.537325 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dea7d6-88c2-444d-bc8f-f010ee53a1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:08.537902 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dea7d6-88c2-444d-bc8f-f010ee53a1eb admin admin] PATH_INFO: `/v3/projects/b559271d692e49c08914742aae23f0f3/users/35c36ac12cf546688684d108f0606922/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:08.577148 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dea7d6-88c2-444d-bc8f-f010ee53a1eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=b559271d692e49c08914742aae23f0f3, user_id=35c36ac12cf546688684d108f0606922, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:08.585173 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dea7d6-88c2-444d-bc8f-f010ee53a1eb admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:08.587539 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1af2ddf-497d-4800-96a8-e059eabea59f admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:08.598829 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee6c09-c15e-44dd-ae2a-c538769d853a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:08.600829 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee6c09-c15e-44dd-ae2a-c538769d853a admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:08.606978 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee6c09-c15e-44dd-ae2a-c538769d853a admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:08.606978 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee6c09-c15e-44dd-ae2a-c538769d853a admin admin] PATH_INFO: `/v3/projects/a8ace4333a914c72a33cbdfd5c8d6835/users/e8d2015ff7144a08aecfd1a2668d3809/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:08.606978 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee6c09-c15e-44dd-ae2a-c538769d853a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8ace4333a914c72a33cbdfd5c8d6835, user_id=e8d2015ff7144a08aecfd1a2668d3809, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:08.608130 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee6c09-c15e-44dd-ae2a-c538769d853a admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:08.611530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:08.623698 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 802/1633] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:08 2022] PUT /identity/v3/projects/a8ace4333a914c72a33cbdfd5c8d6835/users/e8d2015ff7144a08aecfd1a2668d3809/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:08.625907 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6cd6d-a5fa-47ad-8da5-78f0dcdb3a1d None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:08.626402 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6cd6d-a5fa-47ad-8da5-78f0dcdb3a1d None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:08.626783 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6cd6d-a5fa-47ad-8da5-78f0dcdb3a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:08.656975 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-fcb6cd6d-a5fa-47ad-8da5-78f0dcdb3a1d None None] MFA Rules not processed for user `6ced420a85f84a56919ee517c40245a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:08.658399 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-fcb6cd6d-a5fa-47ad-8da5-78f0dcdb3a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:08.702532 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 832/1634] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:08 2022] PUT /identity/v3/projects/b559271d692e49c08914742aae23f0f3/users/35c36ac12cf546688684d108f0606922/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:08.704551 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58a015-e7bc-4328-9855-8e6f3a9b2b6a None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:08.704848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58a015-e7bc-4328-9855-8e6f3a9b2b6a None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:08.705138 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed58a015-e7bc-4328-9855-8e6f3a9b2b6a None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:08.706604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 833/1635] 10.222.0.40 () {54 vars in 912 bytes} [Wed Jul 6 17:13:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:13:08.708832 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe54d4-6a1d-403b-9d7e-5318cfea9dab None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:08.709302 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe54d4-6a1d-403b-9d7e-5318cfea9dab None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:08.709778 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbe54d4-6a1d-403b-9d7e-5318cfea9dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:08.748329 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-3cbe54d4-6a1d-403b-9d7e-5318cfea9dab None None] MFA Rules not processed for user `e8d2015ff7144a08aecfd1a2668d3809`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:08.749907 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-3cbe54d4-6a1d-403b-9d7e-5318cfea9dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:08.849421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 803/1636] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:08 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:08.861873 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb6cd6d-a5fa-47ad-8da5-78f0dcdb3a1d None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:08.913415 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 834/1637] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:08 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:08.915015 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b10e0-2fbe-4cbe-8b7a-038da6a0fedf None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:08.915453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b10e0-2fbe-4cbe-8b7a-038da6a0fedf None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:08.915865 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842b10e0-2fbe-4cbe-8b7a-038da6a0fedf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:08.939478 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-842b10e0-2fbe-4cbe-8b7a-038da6a0fedf None None] MFA Rules not processed for user `af26ef96dc9d47dc91549b736939268e`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:08.941479 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-842b10e0-2fbe-4cbe-8b7a-038da6a0fedf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:08.968166 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:08.982753 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb6cd6d-a5fa-47ad-8da5-78f0dcdb3a1d None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:09.039005 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 835/1638] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 6 17:13:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:09.040482 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b10e0-2fbe-4cbe-8b7a-038da6a0fedf None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:09.062020 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45083b8b-ae3e-4dd0-b757-92cdcf9e365d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:09.062721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45083b8b-ae3e-4dd0-b757-92cdcf9e365d admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:09.063754 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45083b8b-ae3e-4dd0-b757-92cdcf9e365d admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:09.064417 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45083b8b-ae3e-4dd0-b757-92cdcf9e365d admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:09.065906 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45083b8b-ae3e-4dd0-b757-92cdcf9e365d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:09.067528 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45083b8b-ae3e-4dd0-b757-92cdcf9e365d admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:09.086149 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 804/1639] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:09.090205 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45083b8b-ae3e-4dd0-b757-92cdcf9e365d admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:09.106556 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45083b8b-ae3e-4dd0-b757-92cdcf9e365d admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:09.118951 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102514e4-adcb-4fda-adc5-5b19bdeffb6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:09.133170 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842b10e0-2fbe-4cbe-8b7a-038da6a0fedf None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:09.134575 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102514e4-adcb-4fda-adc5-5b19bdeffb6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:09.134870 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102514e4-adcb-4fda-adc5-5b19bdeffb6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:09.135154 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102514e4-adcb-4fda-adc5-5b19bdeffb6a admin admin] PATH_INFO: `/v3/projects/b559271d692e49c08914742aae23f0f3/users/35c36ac12cf546688684d108f0606922/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:09.141875 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102514e4-adcb-4fda-adc5-5b19bdeffb6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b559271d692e49c08914742aae23f0f3, user_id=35c36ac12cf546688684d108f0606922, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:09.143152 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f15a48-e595-4753-8958-484732038079 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:09.144718 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f15a48-e595-4753-8958-484732038079 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:09.145840 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f15a48-e595-4753-8958-484732038079 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:09.146349 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f15a48-e595-4753-8958-484732038079 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:09.150212 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102514e4-adcb-4fda-adc5-5b19bdeffb6a admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:09.190112 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 805/1640] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:09 2022] PUT /identity/v3/projects/b559271d692e49c08914742aae23f0f3/users/35c36ac12cf546688684d108f0606922/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:13:09.192837 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57555387-ac68-4fae-9fde-352da1fc09e8 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:09.193128 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57555387-ac68-4fae-9fde-352da1fc09e8 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:09.193409 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57555387-ac68-4fae-9fde-352da1fc09e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:09.253517 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:09.264373 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f15a48-e595-4753-8958-484732038079 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:09.265896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f15a48-e595-4753-8958-484732038079 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:09.266208 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-57555387-ac68-4fae-9fde-352da1fc09e8 None None] MFA Rules not processed for user `35c36ac12cf546688684d108f0606922`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:09.267770 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-57555387-ac68-4fae-9fde-352da1fc09e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:09.278750 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 836/1641] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:09 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:13:09.279641 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f15a48-e595-4753-8958-484732038079 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:09.290612 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f15a48-e595-4753-8958-484732038079 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:09.423022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 806/1642] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:09 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:13:09.437706 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b41424-039c-4b31-b009-0a3571c03986 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:09.438421 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b41424-039c-4b31-b009-0a3571c03986 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:09.438710 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b41424-039c-4b31-b009-0a3571c03986 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:09.438994 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b41424-039c-4b31-b009-0a3571c03986 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:09.464211 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57555387-ac68-4fae-9fde-352da1fc09e8 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:09.481657 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57555387-ac68-4fae-9fde-352da1fc09e8 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:09.485346 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:09.487929 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f80aa-8177-47d6-aa7e-81e09bd8401e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:09.489454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f80aa-8177-47d6-aa7e-81e09bd8401e service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:09.490512 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f80aa-8177-47d6-aa7e-81e09bd8401e service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:09.491013 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362f80aa-8177-47d6-aa7e-81e09bd8401e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:09.563466 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b41424-039c-4b31-b009-0a3571c03986 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:09.565199 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b41424-039c-4b31-b009-0a3571c03986 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:09.582120 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 837/1643] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:09 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:13:09.616952 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362f80aa-8177-47d6-aa7e-81e09bd8401e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:09.618879 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362f80aa-8177-47d6-aa7e-81e09bd8401e service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:09.631602 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 807/1644] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 17:13:09.723896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b41424-039c-4b31-b009-0a3571c03986 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:09.733580 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b41424-039c-4b31-b009-0a3571c03986 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:09.738857 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97663bb-f561-45cf-9f12-16155212ea63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:09.739667 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97663bb-f561-45cf-9f12-16155212ea63 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:09.740094 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97663bb-f561-45cf-9f12-16155212ea63 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:09.742140 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97663bb-f561-45cf-9f12-16155212ea63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:09.763347 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97663bb-f561-45cf-9f12-16155212ea63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:09.767139 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97663bb-f561-45cf-9f12-16155212ea63 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:09.811794 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 838/1645] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:09 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:13:10.230278 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f80aa-8177-47d6-aa7e-81e09bd8401e service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:10.244228 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362f80aa-8177-47d6-aa7e-81e09bd8401e service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:10.250863 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dbd0d5-10f6-438f-9570-feb657cf02b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:10.251542 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dbd0d5-10f6-438f-9570-feb657cf02b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:10.255812 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dbd0d5-10f6-438f-9570-feb657cf02b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:10.255812 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dbd0d5-10f6-438f-9570-feb657cf02b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:10.368943 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dbd0d5-10f6-438f-9570-feb657cf02b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:10.370907 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dbd0d5-10f6-438f-9570-feb657cf02b4 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:10.385452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 808/1646] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:10 2022] GET /identity/v3/auth/tokens => generated 3220 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:13:10.814500 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97663bb-f561-45cf-9f12-16155212ea63 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:10.956363 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97663bb-f561-45cf-9f12-16155212ea63 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:10.962774 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aebcddf-7428-456f-a380-f6cd1e556f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:10.963757 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aebcddf-7428-456f-a380-f6cd1e556f26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:10.964034 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aebcddf-7428-456f-a380-f6cd1e556f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:10.964312 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aebcddf-7428-456f-a380-f6cd1e556f26 admin admin] PATH_INFO: `/v3/users/f4dc9ce5f5eb48759bd5bf1fdf46d310` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:10.972629 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aebcddf-7428-456f-a380-f6cd1e556f26 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4dc9ce5f5eb48759bd5bf1fdf46d310)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:10.974015 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: WARNING py.warnings [None req-9aebcddf-7428-456f-a380-f6cd1e556f26 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 06 17:13:10.974015 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: warnings.warn(msg) Jul 06 17:13:10.974015 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: Jul 06 17:13:10.975078 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aebcddf-7428-456f-a380-f6cd1e556f26 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:11.237948 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-9aebcddf-7428-456f-a380-f6cd1e556f26 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4dc9ce5f5eb48759bd5bf1fdf46d310'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:11.282929 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-9aebcddf-7428-456f-a380-f6cd1e556f26 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4dc9ce5f5eb48759bd5bf1fdf46d310'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:11.285198 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-9aebcddf-7428-456f-a380-f6cd1e556f26 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4dc9ce5f5eb48759bd5bf1fdf46d310'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:11.317087 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-9aebcddf-7428-456f-a380-f6cd1e556f26 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4dc9ce5f5eb48759bd5bf1fdf46d310'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:11.410056 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-9aebcddf-7428-456f-a380-f6cd1e556f26 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4dc9ce5f5eb48759bd5bf1fdf46d310'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:11.420587 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 839/1647] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:13:10 2022] DELETE /identity/v3/users/f4dc9ce5f5eb48759bd5bf1fdf46d310 => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:11.425205 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67540e89-09c7-4d32-92ed-98a830cce072 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:11.425652 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67540e89-09c7-4d32-92ed-98a830cce072 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:11.425891 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67540e89-09c7-4d32-92ed-98a830cce072 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:11.441114 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2631ac5e-109d-4353-af7d-c4ea44b4d1a8 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:11.441485 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2631ac5e-109d-4353-af7d-c4ea44b4d1a8 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:11.441832 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2631ac5e-109d-4353-af7d-c4ea44b4d1a8 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:11.444188 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 840/1648] 10.222.0.40 () {54 vars in 961 bytes} [Wed Jul 6 17:13:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 17:13:11.455528 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758bb77-12a0-4187-b787-583120bb1948 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:11.461034 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758bb77-12a0-4187-b787-583120bb1948 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:11.461034 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7758bb77-12a0-4187-b787-583120bb1948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:11.475654 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-67540e89-09c7-4d32-92ed-98a830cce072 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:11.476932 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-67540e89-09c7-4d32-92ed-98a830cce072 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:11.477889 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-7758bb77-12a0-4187-b787-583120bb1948 None None] MFA Rules not processed for user `af26ef96dc9d47dc91549b736939268e`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:11.479089 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-7758bb77-12a0-4187-b787-583120bb1948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:11.494967 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:11.518761 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:11.649029 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 809/1649] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:11.671632 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 841/1650] 10.222.0.40 () {58 vars in 1058 bytes} [Wed Jul 6 17:13:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:13:11.671632 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758bb77-12a0-4187-b787-583120bb1948 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:11.671632 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-7758bb77-12a0-4187-b787-583120bb1948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:11.680168 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecfeb25-4c15-44b3-84c9-cb458d30af61 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:11.681414 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecfeb25-4c15-44b3-84c9-cb458d30af61 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:11.681876 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecfeb25-4c15-44b3-84c9-cb458d30af61 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:11.685804 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 810/1651] 10.222.0.40 () {54 vars in 912 bytes} [Wed Jul 6 17:13:11 2022] GET /identity => generated 270 bytes in 26 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:13:11.693117 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca221e12-98c4-47a9-bd95-82b32a332509 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:11.695491 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca221e12-98c4-47a9-bd95-82b32a332509 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:11.696982 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca221e12-98c4-47a9-bd95-82b32a332509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:11.758003 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-ca221e12-98c4-47a9-bd95-82b32a332509 None None] MFA Rules not processed for user `af26ef96dc9d47dc91549b736939268e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:11.759356 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ca221e12-98c4-47a9-bd95-82b32a332509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:11.985688 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 811/1652] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 6 17:13:11 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:13:11.985688 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca221e12-98c4-47a9-bd95-82b32a332509 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:11.985688 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ca221e12-98c4-47a9-bd95-82b32a332509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:11.989333 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7758bb77-12a0-4187-b787-583120bb1948 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:11.989333 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8a94b-8ed3-4691-8d18-a635cc361a50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:11.989333 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8a94b-8ed3-4691-8d18-a635cc361a50 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:11.989333 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8a94b-8ed3-4691-8d18-a635cc361a50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:11.991178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe8a94b-8ed3-4691-8d18-a635cc361a50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:11.991178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe8a94b-8ed3-4691-8d18-a635cc361a50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:11.991178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe8a94b-8ed3-4691-8d18-a635cc361a50 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:11.991178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 842/1653] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:13:11.991178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8a94b-8ed3-4691-8d18-a635cc361a50 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:11.991178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-efe8a94b-8ed3-4691-8d18-a635cc361a50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:11.991932 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca221e12-98c4-47a9-bd95-82b32a332509 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:11.991932 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d244ad-0ba1-4ed6-932e-d8197e406e78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:11.991932 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d244ad-0ba1-4ed6-932e-d8197e406e78 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:11.991932 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d244ad-0ba1-4ed6-932e-d8197e406e78 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:11.992256 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d244ad-0ba1-4ed6-932e-d8197e406e78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:11.992256 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d244ad-0ba1-4ed6-932e-d8197e406e78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:11.992256 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d244ad-0ba1-4ed6-932e-d8197e406e78 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:11.992256 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 812/1654] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 17:13:12.065164 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe8a94b-8ed3-4691-8d18-a635cc361a50 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:12.071267 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5c477-4dea-4de1-a57f-18340eeedfe8 tempest-VolumeQuotaClassesTest-1178586653 tempest-VolumeQuotaClassesTest-1178586653-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35c36ac12cf546688684d108f0606922', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b559271d692e49c08914742aae23f0f3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:12.071993 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5c477-4dea-4de1-a57f-18340eeedfe8 tempest-VolumeQuotaClassesTest-1178586653 tempest-VolumeQuotaClassesTest-1178586653-project-admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:12.072276 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5c477-4dea-4de1-a57f-18340eeedfe8 tempest-VolumeQuotaClassesTest-1178586653 tempest-VolumeQuotaClassesTest-1178586653-project-admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:12.072569 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5c477-4dea-4de1-a57f-18340eeedfe8 tempest-VolumeQuotaClassesTest-1178586653 tempest-VolumeQuotaClassesTest-1178586653-project-admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:12.073313 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5c477-4dea-4de1-a57f-18340eeedfe8 tempest-VolumeQuotaClassesTest-1178586653 tempest-VolumeQuotaClassesTest-1178586653-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:13:12.073626 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5c477-4dea-4de1-a57f-18340eeedfe8 tempest-VolumeQuotaClassesTest-1178586653 tempest-VolumeQuotaClassesTest-1178586653-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:12.075915 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5c477-4dea-4de1-a57f-18340eeedfe8 tempest-VolumeQuotaClassesTest-1178586653 tempest-VolumeQuotaClassesTest-1178586653-project-admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:12.094003 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 843/1655] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:13:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:13:12.097586 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d244ad-0ba1-4ed6-932e-d8197e406e78 service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:12.116901 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d244ad-0ba1-4ed6-932e-d8197e406e78 service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:12.173558 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4bb491-b180-432e-becd-43bb57bfc1d8 tempest-VolumeQuotaClassesTest-1178586653 tempest-VolumeQuotaClassesTest-1178586653-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35c36ac12cf546688684d108f0606922', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b559271d692e49c08914742aae23f0f3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:12.177897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4bb491-b180-432e-becd-43bb57bfc1d8 tempest-VolumeQuotaClassesTest-1178586653 tempest-VolumeQuotaClassesTest-1178586653-project-admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:12.180989 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4bb491-b180-432e-becd-43bb57bfc1d8 tempest-VolumeQuotaClassesTest-1178586653 tempest-VolumeQuotaClassesTest-1178586653-project-admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:12.182194 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4bb491-b180-432e-becd-43bb57bfc1d8 tempest-VolumeQuotaClassesTest-1178586653 tempest-VolumeQuotaClassesTest-1178586653-project-admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:12.191753 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4bb491-b180-432e-becd-43bb57bfc1d8 tempest-VolumeQuotaClassesTest-1178586653 tempest-VolumeQuotaClassesTest-1178586653-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:12.197786 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4bb491-b180-432e-becd-43bb57bfc1d8 tempest-VolumeQuotaClassesTest-1178586653 tempest-VolumeQuotaClassesTest-1178586653-project-admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:12.403272 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 813/1656] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:13:12 2022] POST /identity/v3/projects => generated 355 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:12.408289 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5c477-4dea-4de1-a57f-18340eeedfe8 tempest-VolumeQuotaClassesTest-1178586653 tempest-VolumeQuotaClassesTest-1178586653-project-admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:12.428629 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5c477-4dea-4de1-a57f-18340eeedfe8 tempest-VolumeQuotaClassesTest-1178586653 tempest-VolumeQuotaClassesTest-1178586653-project-admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:12.593903 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12449bd-96d2-437e-98ae-684d3804daac tempest-VolumeQuotaClassesTest-1178586653 tempest-VolumeQuotaClassesTest-1178586653-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35c36ac12cf546688684d108f0606922', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b559271d692e49c08914742aae23f0f3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:12.595144 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12449bd-96d2-437e-98ae-684d3804daac tempest-VolumeQuotaClassesTest-1178586653 tempest-VolumeQuotaClassesTest-1178586653-project-admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:12.595450 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12449bd-96d2-437e-98ae-684d3804daac tempest-VolumeQuotaClassesTest-1178586653 tempest-VolumeQuotaClassesTest-1178586653-project-admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:12.595720 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12449bd-96d2-437e-98ae-684d3804daac tempest-VolumeQuotaClassesTest-1178586653 tempest-VolumeQuotaClassesTest-1178586653-project-admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:12.597031 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12449bd-96d2-437e-98ae-684d3804daac tempest-VolumeQuotaClassesTest-1178586653 tempest-VolumeQuotaClassesTest-1178586653-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:13:12.597467 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12449bd-96d2-437e-98ae-684d3804daac tempest-VolumeQuotaClassesTest-1178586653 tempest-VolumeQuotaClassesTest-1178586653-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:12.600378 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12449bd-96d2-437e-98ae-684d3804daac tempest-VolumeQuotaClassesTest-1178586653 tempest-VolumeQuotaClassesTest-1178586653-project-admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:12.637649 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 844/1657] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:13:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:13:13.085388 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4bb491-b180-432e-becd-43bb57bfc1d8 tempest-VolumeQuotaClassesTest-1178586653 tempest-VolumeQuotaClassesTest-1178586653-project-admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:13.102030 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4bb491-b180-432e-becd-43bb57bfc1d8 tempest-VolumeQuotaClassesTest-1178586653 tempest-VolumeQuotaClassesTest-1178586653-project-admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:13.118879 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c500667-9784-4d19-9ad4-b8732175572d tempest-VolumeQuotaClassesTest-1178586653 tempest-VolumeQuotaClassesTest-1178586653-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35c36ac12cf546688684d108f0606922', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b559271d692e49c08914742aae23f0f3', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:13.120064 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c500667-9784-4d19-9ad4-b8732175572d tempest-VolumeQuotaClassesTest-1178586653 tempest-VolumeQuotaClassesTest-1178586653-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:13.120499 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c500667-9784-4d19-9ad4-b8732175572d tempest-VolumeQuotaClassesTest-1178586653 tempest-VolumeQuotaClassesTest-1178586653-project-admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:13.122855 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c500667-9784-4d19-9ad4-b8732175572d tempest-VolumeQuotaClassesTest-1178586653 tempest-VolumeQuotaClassesTest-1178586653-project-admin] PATH_INFO: `/v3/projects/6019db8b80384846aa0de6190d60d25b` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:13.127542 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c500667-9784-4d19-9ad4-b8732175572d tempest-VolumeQuotaClassesTest-1178586653 tempest-VolumeQuotaClassesTest-1178586653-project-admin] RBAC: Authorizing `identity:delete_project(project_id=6019db8b80384846aa0de6190d60d25b)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:13.131169 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: WARNING py.warnings [None req-0c500667-9784-4d19-9ad4-b8732175572d tempest-VolumeQuotaClassesTest-1178586653 tempest-VolumeQuotaClassesTest-1178586653-project-admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 06 17:13:13.131169 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: warnings.warn(msg) Jul 06 17:13:13.131169 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: Jul 06 17:13:13.132744 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c500667-9784-4d19-9ad4-b8732175572d tempest-VolumeQuotaClassesTest-1178586653 tempest-VolumeQuotaClassesTest-1178586653-project-admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:13.364651 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0c500667-9784-4d19-9ad4-b8732175572d tempest-VolumeQuotaClassesTest-1178586653 tempest-VolumeQuotaClassesTest-1178586653-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6019db8b80384846aa0de6190d60d25b', 'request_id': 'req-0c500667-9784-4d19-9ad4-b8732175572d'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:13.413847 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0c500667-9784-4d19-9ad4-b8732175572d tempest-VolumeQuotaClassesTest-1178586653 tempest-VolumeQuotaClassesTest-1178586653-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6019db8b80384846aa0de6190d60d25b', 'request_id': 'req-0c500667-9784-4d19-9ad4-b8732175572d'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:13.483853 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0c500667-9784-4d19-9ad4-b8732175572d tempest-VolumeQuotaClassesTest-1178586653 tempest-VolumeQuotaClassesTest-1178586653-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6019db8b80384846aa0de6190d60d25b', 'request_id': 'req-0c500667-9784-4d19-9ad4-b8732175572d'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:13.500779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0c500667-9784-4d19-9ad4-b8732175572d tempest-VolumeQuotaClassesTest-1178586653 tempest-VolumeQuotaClassesTest-1178586653-project-admin] The token cache is being invalidate because project 6019db8b80384846aa0de6190d60d25b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:13:13.503583 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0c500667-9784-4d19-9ad4-b8732175572d tempest-VolumeQuotaClassesTest-1178586653 tempest-VolumeQuotaClassesTest-1178586653-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:13.507615 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0c500667-9784-4d19-9ad4-b8732175572d tempest-VolumeQuotaClassesTest-1178586653 tempest-VolumeQuotaClassesTest-1178586653-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:13.508720 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 814/1658] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:13:13 2022] DELETE /identity/v3/projects/6019db8b80384846aa0de6190d60d25b => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:13.713857 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12449bd-96d2-437e-98ae-684d3804daac tempest-VolumeQuotaClassesTest-1178586653 tempest-VolumeQuotaClassesTest-1178586653-project-admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:13.716425 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b12449bd-96d2-437e-98ae-684d3804daac tempest-VolumeQuotaClassesTest-1178586653 tempest-VolumeQuotaClassesTest-1178586653-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:13.925303 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12449bd-96d2-437e-98ae-684d3804daac tempest-VolumeQuotaClassesTest-1178586653 tempest-VolumeQuotaClassesTest-1178586653-project-admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:14.058088 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4ad692-96ea-4b6b-bc22-daba2fae69c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:14.059268 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ad692-96ea-4b6b-bc22-daba2fae69c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:14.059564 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ad692-96ea-4b6b-bc22-daba2fae69c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:14.059874 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4ad692-96ea-4b6b-bc22-daba2fae69c8 admin admin] PATH_INFO: `/v3/projects/e7912de4ef474c2b85310fd847a59bdd` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:14.064612 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4ad692-96ea-4b6b-bc22-daba2fae69c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7912de4ef474c2b85310fd847a59bdd)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:14.066679 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4ad692-96ea-4b6b-bc22-daba2fae69c8 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:14.249135 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0a4ad692-96ea-4b6b-bc22-daba2fae69c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7912de4ef474c2b85310fd847a59bdd', 'request_id': 'req-0a4ad692-96ea-4b6b-bc22-daba2fae69c8'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:14.260421 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0a4ad692-96ea-4b6b-bc22-daba2fae69c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7912de4ef474c2b85310fd847a59bdd', 'request_id': 'req-0a4ad692-96ea-4b6b-bc22-daba2fae69c8'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:14.277387 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0a4ad692-96ea-4b6b-bc22-daba2fae69c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7912de4ef474c2b85310fd847a59bdd', 'request_id': 'req-0a4ad692-96ea-4b6b-bc22-daba2fae69c8'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:14.374998 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0a4ad692-96ea-4b6b-bc22-daba2fae69c8 admin admin] The token cache is being invalidate because project e7912de4ef474c2b85310fd847a59bdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:13:14.374998 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0a4ad692-96ea-4b6b-bc22-daba2fae69c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:14.374998 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0a4ad692-96ea-4b6b-bc22-daba2fae69c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:14.374998 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 845/1659] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:13:13 2022] DELETE /identity/v3/projects/e7912de4ef474c2b85310fd847a59bdd => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:14.378676 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c500667-9784-4d19-9ad4-b8732175572d tempest-VolumeQuotaClassesTest-1178586653 tempest-VolumeQuotaClassesTest-1178586653-project-admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:14.395577 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c500667-9784-4d19-9ad4-b8732175572d tempest-VolumeQuotaClassesTest-1178586653 tempest-VolumeQuotaClassesTest-1178586653-project-admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:14.451089 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:14.519350 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:14.625364 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7ee4c9-feab-4414-8293-6234655df2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:14.626271 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ee4c9-feab-4414-8293-6234655df2dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:14.626723 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ee4c9-feab-4414-8293-6234655df2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:14.627010 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7ee4c9-feab-4414-8293-6234655df2dd admin admin] PATH_INFO: `/v3/users/6ced420a85f84a56919ee517c40245a4` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:14.636748 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7ee4c9-feab-4414-8293-6234655df2dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ced420a85f84a56919ee517c40245a4)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:14.638734 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7ee4c9-feab-4414-8293-6234655df2dd admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:14.900966 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-be7ee4c9-feab-4414-8293-6234655df2dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ced420a85f84a56919ee517c40245a4'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:14.932638 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-be7ee4c9-feab-4414-8293-6234655df2dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ced420a85f84a56919ee517c40245a4'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:14.951189 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-be7ee4c9-feab-4414-8293-6234655df2dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ced420a85f84a56919ee517c40245a4'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:14.954416 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-be7ee4c9-feab-4414-8293-6234655df2dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ced420a85f84a56919ee517c40245a4'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:15.038661 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-be7ee4c9-feab-4414-8293-6234655df2dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ced420a85f84a56919ee517c40245a4'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:15.047077 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 815/1660] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:13:14 2022] DELETE /identity/v3/users/6ced420a85f84a56919ee517c40245a4 => generated 0 bytes in 669 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:13:15.560185 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4ad692-96ea-4b6b-bc22-daba2fae69c8 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:15.567630 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-0a4ad692-96ea-4b6b-bc22-daba2fae69c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:15.616331 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4ad692-96ea-4b6b-bc22-daba2fae69c8 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:15.707237 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:15.884974 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e14a4b-09aa-4fb6-a8e7-5a6b3818b037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:15.886131 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e14a4b-09aa-4fb6-a8e7-5a6b3818b037 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:15.887604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e14a4b-09aa-4fb6-a8e7-5a6b3818b037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:15.888052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e14a4b-09aa-4fb6-a8e7-5a6b3818b037 admin admin] PATH_INFO: `/v3/users/79ef7dc2d5d247b68ab8a9395faa0e40` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:15.892435 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e14a4b-09aa-4fb6-a8e7-5a6b3818b037 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79ef7dc2d5d247b68ab8a9395faa0e40)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:15.898618 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e14a4b-09aa-4fb6-a8e7-5a6b3818b037 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:16.188594 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-21e14a4b-09aa-4fb6-a8e7-5a6b3818b037 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79ef7dc2d5d247b68ab8a9395faa0e40'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:16.202243 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-21e14a4b-09aa-4fb6-a8e7-5a6b3818b037 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79ef7dc2d5d247b68ab8a9395faa0e40'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:16.204000 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-21e14a4b-09aa-4fb6-a8e7-5a6b3818b037 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79ef7dc2d5d247b68ab8a9395faa0e40'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:16.223278 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-21e14a4b-09aa-4fb6-a8e7-5a6b3818b037 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79ef7dc2d5d247b68ab8a9395faa0e40'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:16.351550 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-21e14a4b-09aa-4fb6-a8e7-5a6b3818b037 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79ef7dc2d5d247b68ab8a9395faa0e40'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:16.369968 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 846/1661] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:13:15 2022] DELETE /identity/v3/users/79ef7dc2d5d247b68ab8a9395faa0e40 => generated 0 bytes in 810 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:16.374896 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d29487-a851-4ad7-ae0e-e6da23bd4910 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:16.375476 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d29487-a851-4ad7-ae0e-e6da23bd4910 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:16.375476 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d29487-a851-4ad7-ae0e-e6da23bd4910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:16.419718 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-d0d29487-a851-4ad7-ae0e-e6da23bd4910 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:16.427475 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d0d29487-a851-4ad7-ae0e-e6da23bd4910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:16.585502 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 816/1662] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:16.620099 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e14a4b-09aa-4fb6-a8e7-5a6b3818b037 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:16.622376 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-21e14a4b-09aa-4fb6-a8e7-5a6b3818b037 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:16.757037 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e14a4b-09aa-4fb6-a8e7-5a6b3818b037 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:16.774582 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058b0a54-247b-44e0-bd62-a8788bf08a09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:16.774582 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b0a54-247b-44e0-bd62-a8788bf08a09 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:16.774582 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b0a54-247b-44e0-bd62-a8788bf08a09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:16.774582 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058b0a54-247b-44e0-bd62-a8788bf08a09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:16.791948 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b0a54-247b-44e0-bd62-a8788bf08a09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:16.793887 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058b0a54-247b-44e0-bd62-a8788bf08a09 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:16.816472 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 847/1663] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:13:17.212075 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d29487-a851-4ad7-ae0e-e6da23bd4910 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:17.213703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d0d29487-a851-4ad7-ae0e-e6da23bd4910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:17.232931 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d29487-a851-4ad7-ae0e-e6da23bd4910 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:17.239765 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff274ab-d2b0-4485-87a5-f9f0594b81ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:17.241088 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff274ab-d2b0-4485-87a5-f9f0594b81ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:17.241500 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff274ab-d2b0-4485-87a5-f9f0594b81ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:17.241876 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff274ab-d2b0-4485-87a5-f9f0594b81ca admin admin] PATH_INFO: `/v3/projects/88ebcf54fab546aaa4645834ade22b8a` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:17.243388 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff274ab-d2b0-4485-87a5-f9f0594b81ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=88ebcf54fab546aaa4645834ade22b8a)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:17.245593 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff274ab-d2b0-4485-87a5-f9f0594b81ca admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:17.400392 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:17.428364 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-dff274ab-d2b0-4485-87a5-f9f0594b81ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88ebcf54fab546aaa4645834ade22b8a', 'request_id': 'req-dff274ab-d2b0-4485-87a5-f9f0594b81ca'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:17.446202 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-dff274ab-d2b0-4485-87a5-f9f0594b81ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88ebcf54fab546aaa4645834ade22b8a', 'request_id': 'req-dff274ab-d2b0-4485-87a5-f9f0594b81ca'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:17.529888 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-dff274ab-d2b0-4485-87a5-f9f0594b81ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88ebcf54fab546aaa4645834ade22b8a', 'request_id': 'req-dff274ab-d2b0-4485-87a5-f9f0594b81ca'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:17.548409 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-dff274ab-d2b0-4485-87a5-f9f0594b81ca admin admin] The token cache is being invalidate because project 88ebcf54fab546aaa4645834ade22b8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:13:17.552611 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-dff274ab-d2b0-4485-87a5-f9f0594b81ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:17.552611 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-dff274ab-d2b0-4485-87a5-f9f0594b81ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:17.555848 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 817/1664] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:13:17 2022] DELETE /identity/v3/projects/88ebcf54fab546aaa4645834ade22b8a => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:17.557759 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20955c-f5a5-4021-8193-d9658e6ed5ae None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:17.558202 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20955c-f5a5-4021-8193-d9658e6ed5ae None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:17.558599 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f20955c-f5a5-4021-8193-d9658e6ed5ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:17.614397 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-8f20955c-f5a5-4021-8193-d9658e6ed5ae None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:17.615862 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-8f20955c-f5a5-4021-8193-d9658e6ed5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:17.713119 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 848/1665] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:17.715773 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d19a0-07fe-4121-ad15-d6b67a941c26 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:17.716221 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d19a0-07fe-4121-ad15-d6b67a941c26 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:17.716675 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324d19a0-07fe-4121-ad15-d6b67a941c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:17.756842 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-324d19a0-07fe-4121-ad15-d6b67a941c26 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:17.758268 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-324d19a0-07fe-4121-ad15-d6b67a941c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:17.784169 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 818/1666] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:17.785120 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f20955c-f5a5-4021-8193-d9658e6ed5ae None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:17.803280 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f20955c-f5a5-4021-8193-d9658e6ed5ae None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:17.815576 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6375f7-3f30-4d9c-b036-b49cb63375a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:17.815702 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6375f7-3f30-4d9c-b036-b49cb63375a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:17.815998 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6375f7-3f30-4d9c-b036-b49cb63375a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:17.819032 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6375f7-3f30-4d9c-b036-b49cb63375a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:17.820180 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6375f7-3f30-4d9c-b036-b49cb63375a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:13:17.820327 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6375f7-3f30-4d9c-b036-b49cb63375a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:17.824012 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6375f7-3f30-4d9c-b036-b49cb63375a5 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:17.870836 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 849/1667] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:13:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 17:13:17.882488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d19a0-07fe-4121-ad15-d6b67a941c26 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:17.893147 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324d19a0-07fe-4121-ad15-d6b67a941c26 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:17.904011 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8ed816-96ab-427e-ac93-10dd65c8a393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:17.904883 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ed816-96ab-427e-ac93-10dd65c8a393 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:17.905323 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ed816-96ab-427e-ac93-10dd65c8a393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:17.905751 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8ed816-96ab-427e-ac93-10dd65c8a393 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:17.907154 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8ed816-96ab-427e-ac93-10dd65c8a393 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:17.912072 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8ed816-96ab-427e-ac93-10dd65c8a393 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:18.051921 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6375f7-3f30-4d9c-b036-b49cb63375a5 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:18.065792 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 819/1668] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:13:17 2022] POST /identity/v3/projects => generated 403 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:18.074868 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6375f7-3f30-4d9c-b036-b49cb63375a5 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:18.219276 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53012539-dbc5-4b2d-8d26-632ed43cafca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:18.224464 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53012539-dbc5-4b2d-8d26-632ed43cafca admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:18.224783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53012539-dbc5-4b2d-8d26-632ed43cafca admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:18.225066 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53012539-dbc5-4b2d-8d26-632ed43cafca admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:18.225827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53012539-dbc5-4b2d-8d26-632ed43cafca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:18.232194 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53012539-dbc5-4b2d-8d26-632ed43cafca admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:18.326192 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8ed816-96ab-427e-ac93-10dd65c8a393 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:18.328142 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ea8ed816-96ab-427e-ac93-10dd65c8a393 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:18.352208 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 850/1669] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:13:18 2022] POST /identity/v3/users => generated 302 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:13:18.354967 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53012539-dbc5-4b2d-8d26-632ed43cafca admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:18.355082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8ed816-96ab-427e-ac93-10dd65c8a393 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:18.374489 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53012539-dbc5-4b2d-8d26-632ed43cafca admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:18.382233 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d1dd95-eca5-41e3-b2fe-c6ca7f5da08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:18.386506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1dd95-eca5-41e3-b2fe-c6ca7f5da08d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:18.392062 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1dd95-eca5-41e3-b2fe-c6ca7f5da08d admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:18.392062 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1dd95-eca5-41e3-b2fe-c6ca7f5da08d admin admin] PATH_INFO: `/v3/projects/0ee8f6d39e1d43f297be3cb8e9e5461c` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:18.392062 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d1dd95-eca5-41e3-b2fe-c6ca7f5da08d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ee8f6d39e1d43f297be3cb8e9e5461c)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:18.395099 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2c23f5-f2ed-4146-952e-7cbb01e11079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:18.397188 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c23f5-f2ed-4146-952e-7cbb01e11079 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:18.397799 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c23f5-f2ed-4146-952e-7cbb01e11079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:18.398129 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d1dd95-eca5-41e3-b2fe-c6ca7f5da08d admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:18.398523 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2c23f5-f2ed-4146-952e-7cbb01e11079 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:18.399942 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2c23f5-f2ed-4146-952e-7cbb01e11079 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:18.402419 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2c23f5-f2ed-4146-952e-7cbb01e11079 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:18.426989 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 851/1670] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:18 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:18.432141 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2c23f5-f2ed-4146-952e-7cbb01e11079 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:18.461995 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2c23f5-f2ed-4146-952e-7cbb01e11079 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:18.490049 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb462dd-b8ff-434f-8632-f1585737edbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:18.490049 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb462dd-b8ff-434f-8632-f1585737edbe admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:18.490049 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb462dd-b8ff-434f-8632-f1585737edbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:18.490049 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb462dd-b8ff-434f-8632-f1585737edbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:18.490732 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb462dd-b8ff-434f-8632-f1585737edbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:18.492105 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb462dd-b8ff-434f-8632-f1585737edbe admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:18.512755 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 852/1671] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:18 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:18.514246 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705975a-c34e-4cf6-841b-2912cb9a6a7b None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:18.518894 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705975a-c34e-4cf6-841b-2912cb9a6a7b None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:18.518894 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8705975a-c34e-4cf6-841b-2912cb9a6a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:18.562575 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-8705975a-c34e-4cf6-841b-2912cb9a6a7b None None] MFA Rules not processed for user `7922db6ad0ce41868a68f5a834a3b8f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:18.570722 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-8705975a-c34e-4cf6-841b-2912cb9a6a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:18.594402 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-83d1dd95-eca5-41e3-b2fe-c6ca7f5da08d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ee8f6d39e1d43f297be3cb8e9e5461c', 'request_id': 'req-83d1dd95-eca5-41e3-b2fe-c6ca7f5da08d'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:18.595226 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:18.623549 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-83d1dd95-eca5-41e3-b2fe-c6ca7f5da08d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ee8f6d39e1d43f297be3cb8e9e5461c', 'request_id': 'req-83d1dd95-eca5-41e3-b2fe-c6ca7f5da08d'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:18.712180 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-83d1dd95-eca5-41e3-b2fe-c6ca7f5da08d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ee8f6d39e1d43f297be3cb8e9e5461c', 'request_id': 'req-83d1dd95-eca5-41e3-b2fe-c6ca7f5da08d'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:18.741326 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-83d1dd95-eca5-41e3-b2fe-c6ca7f5da08d admin admin] The token cache is being invalidate because project 0ee8f6d39e1d43f297be3cb8e9e5461c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:13:18.741914 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-83d1dd95-eca5-41e3-b2fe-c6ca7f5da08d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:18.742795 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-83d1dd95-eca5-41e3-b2fe-c6ca7f5da08d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:18.745221 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 820/1672] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:13:18 2022] DELETE /identity/v3/projects/0ee8f6d39e1d43f297be3cb8e9e5461c => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:13:18.753799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d1dd95-eca5-41e3-b2fe-c6ca7f5da08d admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:18.753799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-83d1dd95-eca5-41e3-b2fe-c6ca7f5da08d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:18.830874 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 853/1673] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:18.831894 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8705975a-c34e-4cf6-841b-2912cb9a6a7b None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:18.843443 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d1dd95-eca5-41e3-b2fe-c6ca7f5da08d admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:18.845972 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-8705975a-c34e-4cf6-841b-2912cb9a6a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:18.849551 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442414df-20f0-499f-8cbf-b6baf9ab3042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:18.857834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442414df-20f0-499f-8cbf-b6baf9ab3042 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:18.857834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442414df-20f0-499f-8cbf-b6baf9ab3042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:18.857834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442414df-20f0-499f-8cbf-b6baf9ab3042 admin admin] PATH_INFO: `/v3/projects/e703c86373a64c1386a02f301eb1bbbf/users/a8c6e03fe8c14d5899eadc2b26e9c44c/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:18.870380 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442414df-20f0-499f-8cbf-b6baf9ab3042 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e703c86373a64c1386a02f301eb1bbbf, user_id=a8c6e03fe8c14d5899eadc2b26e9c44c, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:18.874409 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442414df-20f0-499f-8cbf-b6baf9ab3042 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:18.896195 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8705975a-c34e-4cf6-841b-2912cb9a6a7b None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:18.912413 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd1f1c4-e1ef-4f6f-a14a-a6158bbf8e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:18.913255 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1f1c4-e1ef-4f6f-a14a-a6158bbf8e61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:18.913537 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1f1c4-e1ef-4f6f-a14a-a6158bbf8e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:18.913813 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd1f1c4-e1ef-4f6f-a14a-a6158bbf8e61 admin admin] PATH_INFO: `/v3/users/35c36ac12cf546688684d108f0606922` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:18.917676 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd1f1c4-e1ef-4f6f-a14a-a6158bbf8e61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35c36ac12cf546688684d108f0606922)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:18.919798 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd1f1c4-e1ef-4f6f-a14a-a6158bbf8e61 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:18.968265 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 821/1674] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:18 2022] PUT /identity/v3/projects/e703c86373a64c1386a02f301eb1bbbf/users/a8c6e03fe8c14d5899eadc2b26e9c44c/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:18.969550 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442414df-20f0-499f-8cbf-b6baf9ab3042 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:18.975467 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-442414df-20f0-499f-8cbf-b6baf9ab3042 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:19.180390 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442414df-20f0-499f-8cbf-b6baf9ab3042 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:19.193802 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9d6336-b708-4f32-92f4-58523026172f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:19.194670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d6336-b708-4f32-92f4-58523026172f service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:19.195102 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d6336-b708-4f32-92f4-58523026172f service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:19.195525 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9d6336-b708-4f32-92f4-58523026172f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:19.236067 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-6fd1f1c4-e1ef-4f6f-a14a-a6158bbf8e61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35c36ac12cf546688684d108f0606922'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:19.251157 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-6fd1f1c4-e1ef-4f6f-a14a-a6158bbf8e61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35c36ac12cf546688684d108f0606922'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:19.253631 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-6fd1f1c4-e1ef-4f6f-a14a-a6158bbf8e61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35c36ac12cf546688684d108f0606922'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:19.270224 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9d6336-b708-4f32-92f4-58523026172f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:19.272197 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9d6336-b708-4f32-92f4-58523026172f service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:19.283676 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-6fd1f1c4-e1ef-4f6f-a14a-a6158bbf8e61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35c36ac12cf546688684d108f0606922'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:19.289705 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 822/1675] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:13:19.300315 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c92fd6-d810-4470-9afe-f393a9740467 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:19.300315 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c92fd6-d810-4470-9afe-f393a9740467 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:19.300315 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c92fd6-d810-4470-9afe-f393a9740467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:19.320019 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-36c92fd6-d810-4470-9afe-f393a9740467 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:19.321594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-36c92fd6-d810-4470-9afe-f393a9740467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:19.404024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-6fd1f1c4-e1ef-4f6f-a14a-a6158bbf8e61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35c36ac12cf546688684d108f0606922'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:19.412157 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 854/1676] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:13:18 2022] DELETE /identity/v3/users/35c36ac12cf546688684d108f0606922 => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:19.593534 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 823/1677] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:19.624766 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd1f1c4-e1ef-4f6f-a14a-a6158bbf8e61 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:19.626762 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-6fd1f1c4-e1ef-4f6f-a14a-a6158bbf8e61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:19.735918 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd1f1c4-e1ef-4f6f-a14a-a6158bbf8e61 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:19.754281 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590de8ee-60cd-4680-a0f4-20e7b1d375a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:19.755586 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590de8ee-60cd-4680-a0f4-20e7b1d375a0 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:19.756818 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590de8ee-60cd-4680-a0f4-20e7b1d375a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:19.766232 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590de8ee-60cd-4680-a0f4-20e7b1d375a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:19.967235 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590de8ee-60cd-4680-a0f4-20e7b1d375a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:19.971004 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590de8ee-60cd-4680-a0f4-20e7b1d375a0 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:19.993412 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 855/1678] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:13:20.004407 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=84,keystone:DELETE=12,keystone:INSERT=4 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:20.236701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956bc572-1515-4984-80d4-67d0f7f13e01 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:20.237160 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956bc572-1515-4984-80d4-67d0f7f13e01 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:20.238193 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956bc572-1515-4984-80d4-67d0f7f13e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:20.311572 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-956bc572-1515-4984-80d4-67d0f7f13e01 None None] MFA Rules not processed for user `8e302cf8d56c4db8bb8480ec234ac908`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:20.312900 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-956bc572-1515-4984-80d4-67d0f7f13e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:20.485786 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 824/1679] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:20.496422 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1bccb-a80b-4d95-b413-3024cd40f816 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:20.496934 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1bccb-a80b-4d95-b413-3024cd40f816 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:20.497332 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1bccb-a80b-4d95-b413-3024cd40f816 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:20.545145 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-4bc1bccb-a80b-4d95-b413-3024cd40f816 None None] MFA Rules not processed for user `8e302cf8d56c4db8bb8480ec234ac908`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:20.548877 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-4bc1bccb-a80b-4d95-b413-3024cd40f816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:20.568982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 856/1680] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:20 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:20.587232 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956bc572-1515-4984-80d4-67d0f7f13e01 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:20.589962 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1bccb-a80b-4d95-b413-3024cd40f816 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:20.590626 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-956bc572-1515-4984-80d4-67d0f7f13e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:20.591364 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-4bc1bccb-a80b-4d95-b413-3024cd40f816 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:20.619934 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1bccb-a80b-4d95-b413-3024cd40f816 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:20.633375 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d630b806-4256-4894-93e8-70031fc93afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:20.633375 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d630b806-4256-4894-93e8-70031fc93afd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:20.633375 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d630b806-4256-4894-93e8-70031fc93afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:20.633375 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d630b806-4256-4894-93e8-70031fc93afd admin admin] PATH_INFO: `/v3/projects/b559271d692e49c08914742aae23f0f3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:20.641937 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d630b806-4256-4894-93e8-70031fc93afd admin admin] RBAC: Authorizing `identity:delete_project(project_id=b559271d692e49c08914742aae23f0f3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:20.644007 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d630b806-4256-4894-93e8-70031fc93afd admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:20.805475 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956bc572-1515-4984-80d4-67d0f7f13e01 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:20.815847 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72239084-1763-4602-9f00-104f9b76b68a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:20.821064 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72239084-1763-4602-9f00-104f9b76b68a service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:20.821064 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72239084-1763-4602-9f00-104f9b76b68a service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:20.821064 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72239084-1763-4602-9f00-104f9b76b68a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:20.846728 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72239084-1763-4602-9f00-104f9b76b68a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:20.848616 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72239084-1763-4602-9f00-104f9b76b68a service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:20.874460 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 825/1681] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:20 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:13:20.931654 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:20.941537 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-d630b806-4256-4894-93e8-70031fc93afd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b559271d692e49c08914742aae23f0f3', 'request_id': 'req-d630b806-4256-4894-93e8-70031fc93afd'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:20.959706 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-d630b806-4256-4894-93e8-70031fc93afd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b559271d692e49c08914742aae23f0f3', 'request_id': 'req-d630b806-4256-4894-93e8-70031fc93afd'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:20.976369 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-d630b806-4256-4894-93e8-70031fc93afd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b559271d692e49c08914742aae23f0f3', 'request_id': 'req-d630b806-4256-4894-93e8-70031fc93afd'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:21.088483 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-d630b806-4256-4894-93e8-70031fc93afd admin admin] The token cache is being invalidate because project b559271d692e49c08914742aae23f0f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:13:21.088483 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-d630b806-4256-4894-93e8-70031fc93afd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:21.088483 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-d630b806-4256-4894-93e8-70031fc93afd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:21.095803 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 857/1682] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:13:20 2022] DELETE /identity/v3/projects/b559271d692e49c08914742aae23f0f3 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:21.101651 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9ef96e-eec0-4ea4-9ec2-32b54bef308b None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:21.102202 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9ef96e-eec0-4ea4-9ec2-32b54bef308b None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:21.102202 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9ef96e-eec0-4ea4-9ec2-32b54bef308b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:21.156889 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-0e9ef96e-eec0-4ea4-9ec2-32b54bef308b None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:21.158215 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-0e9ef96e-eec0-4ea4-9ec2-32b54bef308b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:21.323946 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 826/1683] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:21.333756 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c7081-3744-4c53-8932-a8f49a2440ec None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:21.334268 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c7081-3744-4c53-8932-a8f49a2440ec None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:21.334562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2c7081-3744-4c53-8932-a8f49a2440ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:21.358241 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-db2c7081-3744-4c53-8932-a8f49a2440ec None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:21.359923 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-db2c7081-3744-4c53-8932-a8f49a2440ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:21.374534 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 858/1684] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:21.380174 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9ef96e-eec0-4ea4-9ec2-32b54bef308b None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:21.389960 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9ef96e-eec0-4ea4-9ec2-32b54bef308b None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:21.424616 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2524b4d7-8d3c-4589-81cd-87ff88e14a61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:21.425526 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524b4d7-8d3c-4589-81cd-87ff88e14a61 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:21.425820 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524b4d7-8d3c-4589-81cd-87ff88e14a61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:21.426202 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524b4d7-8d3c-4589-81cd-87ff88e14a61 admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:21.426837 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2524b4d7-8d3c-4589-81cd-87ff88e14a61 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:13:21.427398 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2524b4d7-8d3c-4589-81cd-87ff88e14a61 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:21.429298 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2524b4d7-8d3c-4589-81cd-87ff88e14a61 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:21.473849 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c7081-3744-4c53-8932-a8f49a2440ec None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:21.473849 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-db2c7081-3744-4c53-8932-a8f49a2440ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:21.502116 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2c7081-3744-4c53-8932-a8f49a2440ec None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:21.504561 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 827/1685] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:13:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:13:21.509803 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2524b4d7-8d3c-4589-81cd-87ff88e14a61 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:21.513428 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e3d5a9-e635-405c-8b0f-a86035934830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:21.516638 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3d5a9-e635-405c-8b0f-a86035934830 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:21.516638 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3d5a9-e635-405c-8b0f-a86035934830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:21.516638 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e3d5a9-e635-405c-8b0f-a86035934830 admin admin] PATH_INFO: `/v3/users/5ace6486a5d942e08f3f4708a4193e4a` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:21.519706 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e3d5a9-e635-405c-8b0f-a86035934830 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ace6486a5d942e08f3f4708a4193e4a)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:21.527874 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e3d5a9-e635-405c-8b0f-a86035934830 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:21.563273 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2524b4d7-8d3c-4589-81cd-87ff88e14a61 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:21.579635 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff8fe4-982c-49c1-8e00-009fc981e950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:21.580949 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff8fe4-982c-49c1-8e00-009fc981e950 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:21.581313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff8fe4-982c-49c1-8e00-009fc981e950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:21.581734 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff8fe4-982c-49c1-8e00-009fc981e950 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:21.582565 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ff8fe4-982c-49c1-8e00-009fc981e950 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:21.589567 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ff8fe4-982c-49c1-8e00-009fc981e950 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:21.729914 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 828/1686] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:13:21 2022] POST /identity/v3/projects => generated 379 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:21.736147 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff8fe4-982c-49c1-8e00-009fc981e950 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:21.803380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-02e3d5a9-e635-405c-8b0f-a86035934830 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ace6486a5d942e08f3f4708a4193e4a'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:21.829002 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-02e3d5a9-e635-405c-8b0f-a86035934830 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ace6486a5d942e08f3f4708a4193e4a'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:21.829997 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-02e3d5a9-e635-405c-8b0f-a86035934830 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ace6486a5d942e08f3f4708a4193e4a'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:21.845599 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-02e3d5a9-e635-405c-8b0f-a86035934830 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ace6486a5d942e08f3f4708a4193e4a'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:21.912749 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff8fe4-982c-49c1-8e00-009fc981e950 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:21.917730 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-02e3d5a9-e635-405c-8b0f-a86035934830 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ace6486a5d942e08f3f4708a4193e4a'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:21.924813 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 859/1687] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:13:21 2022] DELETE /identity/v3/users/5ace6486a5d942e08f3f4708a4193e4a => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:21.929966 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5d8dc-c5e4-4982-ba8c-d05719996998 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:21.930455 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5d8dc-c5e4-4982-ba8c-d05719996998 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:21.931326 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db5d8dc-c5e4-4982-ba8c-d05719996998 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:21.935297 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 860/1688] 10.222.0.40 () {54 vars in 912 bytes} [Wed Jul 6 17:13:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:13:21.942164 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71eab72-b4f2-47e2-9526-65bc2b504c56 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:21.943427 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71eab72-b4f2-47e2-9526-65bc2b504c56 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:21.943960 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71eab72-b4f2-47e2-9526-65bc2b504c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:21.946383 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e780813b-0066-4ba1-8658-33f444c584a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:21.960935 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e780813b-0066-4ba1-8658-33f444c584a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:21.963014 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e780813b-0066-4ba1-8658-33f444c584a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:21.963501 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e780813b-0066-4ba1-8658-33f444c584a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:21.968022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e780813b-0066-4ba1-8658-33f444c584a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:21.971697 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e780813b-0066-4ba1-8658-33f444c584a3 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:22.008676 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-d71eab72-b4f2-47e2-9526-65bc2b504c56 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:22.010935 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d71eab72-b4f2-47e2-9526-65bc2b504c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:22.107880 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 829/1689] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:13:21 2022] POST /identity/v3/users => generated 290 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:22.109806 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c245f392-55e6-4d2e-bbac-7e63267675d8 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:22.110279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c245f392-55e6-4d2e-bbac-7e63267675d8 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:22.110564 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c245f392-55e6-4d2e-bbac-7e63267675d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:22.159386 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 861/1690] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:22.159571 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911029d-bff3-42e6-852a-ba07d050db74 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:22.163403 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911029d-bff3-42e6-852a-ba07d050db74 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:22.163403 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5911029d-bff3-42e6-852a-ba07d050db74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:22.214637 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:22.224874 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-c245f392-55e6-4d2e-bbac-7e63267675d8 None None] MFA Rules not processed for user `5f479458a68b4daf9f3a7ae53d6c3763`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:22.226160 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c245f392-55e6-4d2e-bbac-7e63267675d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:22.234363 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-5911029d-bff3-42e6-852a-ba07d050db74 None None] MFA Rules not processed for user `8e72875df16d497f9a542da0a5636f5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:22.238165 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-5911029d-bff3-42e6-852a-ba07d050db74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:22.380978 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 862/1691] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:22.381977 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5911029d-bff3-42e6-852a-ba07d050db74 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:22.397859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-5911029d-bff3-42e6-852a-ba07d050db74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:22.406850 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 830/1692] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 6 17:13:22 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:22.410090 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c245f392-55e6-4d2e-bbac-7e63267675d8 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:22.420316 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c245f392-55e6-4d2e-bbac-7e63267675d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:22.440558 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5911029d-bff3-42e6-852a-ba07d050db74 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:22.441491 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c245f392-55e6-4d2e-bbac-7e63267675d8 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:22.449202 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be19571-aeb3-4a18-afaa-e0f49b45d705 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:22.449751 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be19571-aeb3-4a18-afaa-e0f49b45d705 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:22.450081 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be19571-aeb3-4a18-afaa-e0f49b45d705 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:22.450311 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be19571-aeb3-4a18-afaa-e0f49b45d705 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:22.451097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e57350-3967-4e4c-b8bf-2717619245c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:22.451806 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e57350-3967-4e4c-b8bf-2717619245c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:22.452410 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e57350-3967-4e4c-b8bf-2717619245c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:22.452410 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e57350-3967-4e4c-b8bf-2717619245c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:22.454387 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e57350-3967-4e4c-b8bf-2717619245c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:22.456066 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e57350-3967-4e4c-b8bf-2717619245c6 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:22.474941 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be19571-aeb3-4a18-afaa-e0f49b45d705 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:22.476452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be19571-aeb3-4a18-afaa-e0f49b45d705 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:22.478167 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 863/1693] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:22 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:22.480018 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a902b00-2f89-4b9e-a4bc-fc984ba9f588 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:22.480502 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a902b00-2f89-4b9e-a4bc-fc984ba9f588 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:22.481024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a902b00-2f89-4b9e-a4bc-fc984ba9f588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:22.510259 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-0a902b00-2f89-4b9e-a4bc-fc984ba9f588 None None] MFA Rules not processed for user `8e72875df16d497f9a542da0a5636f5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:22.511684 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-0a902b00-2f89-4b9e-a4bc-fc984ba9f588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:22.514884 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 831/1694] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:13:22.516472 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be19571-aeb3-4a18-afaa-e0f49b45d705 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:22.521547 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 864/1695] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:22.524353 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a902b00-2f89-4b9e-a4bc-fc984ba9f588 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:22.541752 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a902b00-2f89-4b9e-a4bc-fc984ba9f588 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:22.549524 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be19571-aeb3-4a18-afaa-e0f49b45d705 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:22.551489 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce117b94-1df3-489d-9f7b-69fa0a0456a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:22.552404 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce117b94-1df3-489d-9f7b-69fa0a0456a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:22.552888 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce117b94-1df3-489d-9f7b-69fa0a0456a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:22.553712 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce117b94-1df3-489d-9f7b-69fa0a0456a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:22.554656 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce117b94-1df3-489d-9f7b-69fa0a0456a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:22.559582 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce117b94-1df3-489d-9f7b-69fa0a0456a9 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:22.584071 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232fdb49-0582-456a-b8d1-888058ffdc40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:22.584071 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232fdb49-0582-456a-b8d1-888058ffdc40 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:22.584071 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232fdb49-0582-456a-b8d1-888058ffdc40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:22.584071 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232fdb49-0582-456a-b8d1-888058ffdc40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:22.603477 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 865/1696] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:22 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:22.604916 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce117b94-1df3-489d-9f7b-69fa0a0456a9 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:22.609919 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-ce117b94-1df3-489d-9f7b-69fa0a0456a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:22.615448 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232fdb49-0582-456a-b8d1-888058ffdc40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:22.624057 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232fdb49-0582-456a-b8d1-888058ffdc40 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:22.637554 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 832/1697] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:13:22.639594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232fdb49-0582-456a-b8d1-888058ffdc40 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:22.671550 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232fdb49-0582-456a-b8d1-888058ffdc40 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:22.696303 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31b3f91-388d-4d05-96a3-47b3a63ad66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:22.697268 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b3f91-388d-4d05-96a3-47b3a63ad66a admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:22.697560 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b3f91-388d-4d05-96a3-47b3a63ad66a admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:22.697844 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31b3f91-388d-4d05-96a3-47b3a63ad66a admin admin] PATH_INFO: `/v3/projects/58ff8d20419140388a4e0baf04f66ddb/users/33ba5acd4dc54475b2eb0489083006a0/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:22.729656 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31b3f91-388d-4d05-96a3-47b3a63ad66a admin admin] RBAC: Authorizing `identity:create_grant(project_id=58ff8d20419140388a4e0baf04f66ddb, user_id=33ba5acd4dc54475b2eb0489083006a0, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:22.732356 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31b3f91-388d-4d05-96a3-47b3a63ad66a admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:22.830001 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce117b94-1df3-489d-9f7b-69fa0a0456a9 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:22.836069 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4132ab10-8980-4023-8661-70b08b49219a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:22.836891 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4132ab10-8980-4023-8661-70b08b49219a service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:22.837304 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4132ab10-8980-4023-8661-70b08b49219a service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:22.848500 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4132ab10-8980-4023-8661-70b08b49219a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:22.868331 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4132ab10-8980-4023-8661-70b08b49219a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:22.869782 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4132ab10-8980-4023-8661-70b08b49219a service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:22.899491 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:22.903395 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 866/1698] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:22 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:13:22.907088 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36ef09-ecb6-455e-b026-444b36bf262b None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:22.907652 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36ef09-ecb6-455e-b026-444b36bf262b None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:22.908776 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36ef09-ecb6-455e-b026-444b36bf262b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:22.915877 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 833/1699] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:22 2022] PUT /identity/v3/projects/58ff8d20419140388a4e0baf04f66ddb/users/33ba5acd4dc54475b2eb0489083006a0/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:22.928903 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0156c9-96ae-4203-bcb6-fa75f8d37ab7 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:22.929590 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0156c9-96ae-4203-bcb6-fa75f8d37ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:22.930039 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0156c9-96ae-4203-bcb6-fa75f8d37ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:22.969684 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:22.992727 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-ff36ef09-ecb6-455e-b026-444b36bf262b None None] MFA Rules not processed for user `a8c6e03fe8c14d5899eadc2b26e9c44c`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:22.994161 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-ff36ef09-ecb6-455e-b026-444b36bf262b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:23.049650 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-6a0156c9-96ae-4203-bcb6-fa75f8d37ab7 None None] MFA Rules not processed for user `33ba5acd4dc54475b2eb0489083006a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:23.054298 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-6a0156c9-96ae-4203-bcb6-fa75f8d37ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:23.214512 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 867/1700] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:22 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:23.215763 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff36ef09-ecb6-455e-b026-444b36bf262b None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:23.217166 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-ff36ef09-ecb6-455e-b026-444b36bf262b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:23.260443 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 834/1701] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:23.275941 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0156c9-96ae-4203-bcb6-fa75f8d37ab7 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:23.281596 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-6a0156c9-96ae-4203-bcb6-fa75f8d37ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:23.371104 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff36ef09-ecb6-455e-b026-444b36bf262b None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:23.384240 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90790a-06d0-419f-9d3f-b760aa250053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:23.386776 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90790a-06d0-419f-9d3f-b760aa250053 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:23.387851 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90790a-06d0-419f-9d3f-b760aa250053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:23.388893 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90790a-06d0-419f-9d3f-b760aa250053 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:23.389817 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee90790a-06d0-419f-9d3f-b760aa250053 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:23.396175 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee90790a-06d0-419f-9d3f-b760aa250053 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:23.465353 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0156c9-96ae-4203-bcb6-fa75f8d37ab7 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:23.474934 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82846e02-a396-4a2d-b808-2aedfc93c5c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:23.475869 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82846e02-a396-4a2d-b808-2aedfc93c5c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:23.476284 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82846e02-a396-4a2d-b808-2aedfc93c5c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:23.476670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82846e02-a396-4a2d-b808-2aedfc93c5c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:23.487419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82846e02-a396-4a2d-b808-2aedfc93c5c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:23.489225 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82846e02-a396-4a2d-b808-2aedfc93c5c0 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:23.511384 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 835/1702] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:23 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:13:23.521554 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82846e02-a396-4a2d-b808-2aedfc93c5c0 service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:23.538027 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82846e02-a396-4a2d-b808-2aedfc93c5c0 service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:23.550708 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6850a9ea-f51b-4e43-bb26-220ea4b70a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:23.552134 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850a9ea-f51b-4e43-bb26-220ea4b70a20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:23.552469 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850a9ea-f51b-4e43-bb26-220ea4b70a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:23.552905 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6850a9ea-f51b-4e43-bb26-220ea4b70a20 admin admin] PATH_INFO: `/v3/users/33ba5acd4dc54475b2eb0489083006a0` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:23.554446 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6850a9ea-f51b-4e43-bb26-220ea4b70a20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33ba5acd4dc54475b2eb0489083006a0)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:23.557323 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6850a9ea-f51b-4e43-bb26-220ea4b70a20 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:23.599810 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 868/1703] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:13:23 2022] POST /identity/v3/projects => generated 405 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:23.601166 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90790a-06d0-419f-9d3f-b760aa250053 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:23.614641 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-ee90790a-06d0-419f-9d3f-b760aa250053 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:23.848976 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90790a-06d0-419f-9d3f-b760aa250053 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:23.855427 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67256288-03be-45b8-8256-905f31ab1e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:23.856391 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67256288-03be-45b8-8256-905f31ab1e42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:23.856795 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67256288-03be-45b8-8256-905f31ab1e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:23.857096 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67256288-03be-45b8-8256-905f31ab1e42 admin admin] PATH_INFO: `/v3/projects/71698505c2fa4176a7dbcb62ef137649` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:23.858807 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67256288-03be-45b8-8256-905f31ab1e42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71698505c2fa4176a7dbcb62ef137649)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:23.861417 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67256288-03be-45b8-8256-905f31ab1e42 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:23.941618 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6850a9ea-f51b-4e43-bb26-220ea4b70a20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33ba5acd4dc54475b2eb0489083006a0'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:24.020517 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6850a9ea-f51b-4e43-bb26-220ea4b70a20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33ba5acd4dc54475b2eb0489083006a0'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:24.034091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6850a9ea-f51b-4e43-bb26-220ea4b70a20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33ba5acd4dc54475b2eb0489083006a0'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:24.035548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6850a9ea-f51b-4e43-bb26-220ea4b70a20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33ba5acd4dc54475b2eb0489083006a0'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:24.142901 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:24.144665 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6850a9ea-f51b-4e43-bb26-220ea4b70a20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33ba5acd4dc54475b2eb0489083006a0'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:24.153778 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 836/1704] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:13:23 2022] DELETE /identity/v3/users/33ba5acd4dc54475b2eb0489083006a0 => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:13:24.154892 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6850a9ea-f51b-4e43-bb26-220ea4b70a20 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:24.157701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-6850a9ea-f51b-4e43-bb26-220ea4b70a20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:24.161805 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-67256288-03be-45b8-8256-905f31ab1e42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71698505c2fa4176a7dbcb62ef137649', 'request_id': 'req-67256288-03be-45b8-8256-905f31ab1e42'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:24.170913 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-67256288-03be-45b8-8256-905f31ab1e42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71698505c2fa4176a7dbcb62ef137649', 'request_id': 'req-67256288-03be-45b8-8256-905f31ab1e42'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:24.201478 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-67256288-03be-45b8-8256-905f31ab1e42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71698505c2fa4176a7dbcb62ef137649', 'request_id': 'req-67256288-03be-45b8-8256-905f31ab1e42'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:24.323288 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-67256288-03be-45b8-8256-905f31ab1e42 admin admin] The token cache is being invalidate because project 71698505c2fa4176a7dbcb62ef137649 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:13:24.330900 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-67256288-03be-45b8-8256-905f31ab1e42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:24.334071 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-67256288-03be-45b8-8256-905f31ab1e42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:24.335308 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6850a9ea-f51b-4e43-bb26-220ea4b70a20 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:24.336186 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 869/1705] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:13:23 2022] DELETE /identity/v3/projects/71698505c2fa4176a7dbcb62ef137649 => generated 0 bytes in 735 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:24.337605 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047b886-5ae3-4077-9e9d-acd37b077330 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:24.337958 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047b886-5ae3-4077-9e9d-acd37b077330 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:24.338772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3047b886-5ae3-4077-9e9d-acd37b077330 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:24.340654 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 870/1706] 10.222.0.40 () {54 vars in 960 bytes} [Wed Jul 6 17:13:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:13:24.342010 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b066eb-06c4-4254-bf7e-419b4d4256df None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:24.342458 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b066eb-06c4-4254-bf7e-419b4d4256df None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:24.342967 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b066eb-06c4-4254-bf7e-419b4d4256df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:24.357868 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5ffb84-1754-4a14-9fd1-667c5f38bfab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:24.358845 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ffb84-1754-4a14-9fd1-667c5f38bfab admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:24.361983 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ffb84-1754-4a14-9fd1-667c5f38bfab admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:24.370993 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5ffb84-1754-4a14-9fd1-667c5f38bfab admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:24.370993 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5ffb84-1754-4a14-9fd1-667c5f38bfab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:24.370993 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5ffb84-1754-4a14-9fd1-667c5f38bfab admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:24.400898 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-82b066eb-06c4-4254-bf7e-419b4d4256df None None] MFA Rules not processed for user `8e302cf8d56c4db8bb8480ec234ac908`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:24.406676 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-82b066eb-06c4-4254-bf7e-419b4d4256df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:24.479295 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 871/1707] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:24 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:24.485347 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c653c79-fa8b-40af-bd00-651dbbd2fb9e None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:24.488171 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c653c79-fa8b-40af-bd00-651dbbd2fb9e None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:24.491526 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c653c79-fa8b-40af-bd00-651dbbd2fb9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:24.492843 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 837/1708] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:13:24 2022] POST /identity/v3/users => generated 309 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:24.494467 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f299c-814e-46bc-966b-287ef9cb08d4 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:24.494943 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f299c-814e-46bc-966b-287ef9cb08d4 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:24.495559 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f299c-814e-46bc-966b-287ef9cb08d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:24.533597 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-8c653c79-fa8b-40af-bd00-651dbbd2fb9e None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:24.535220 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-8c653c79-fa8b-40af-bd00-651dbbd2fb9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:24.536756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-3c7f299c-814e-46bc-966b-287ef9cb08d4 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:24.537966 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-3c7f299c-814e-46bc-966b-287ef9cb08d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:24.559327 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 872/1709] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:24.559327 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb4423-fd44-413d-8b80-fbda6d7b2cea None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:24.559327 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb4423-fd44-413d-8b80-fbda6d7b2cea None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:24.559327 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddb4423-fd44-413d-8b80-fbda6d7b2cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:24.565703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 838/1710] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:24.566867 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2b36f-fa64-4332-9982-c0481d2cc56b None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:24.570752 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2b36f-fa64-4332-9982-c0481d2cc56b None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:24.571053 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2b36f-fa64-4332-9982-c0481d2cc56b None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:24.573051 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 839/1711] 10.222.0.40 () {54 vars in 912 bytes} [Wed Jul 6 17:13:24 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:13:24.577119 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa2b36f-fa64-4332-9982-c0481d2cc56b None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:24.580798 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-7aa2b36f-fa64-4332-9982-c0481d2cc56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:24.595436 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-2ddb4423-fd44-413d-8b80-fbda6d7b2cea None None] MFA Rules not processed for user `76ca877bee9041d28ad41e45ef3dd18c`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:24.595436 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-2ddb4423-fd44-413d-8b80-fbda6d7b2cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:24.619069 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa2b36f-fa64-4332-9982-c0481d2cc56b None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:24.635721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5074d440-0c80-45a6-8092-713f8c4283bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:24.640947 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074d440-0c80-45a6-8092-713f8c4283bc admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:24.641235 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074d440-0c80-45a6-8092-713f8c4283bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:24.641723 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5074d440-0c80-45a6-8092-713f8c4283bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:24.642758 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5074d440-0c80-45a6-8092-713f8c4283bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:24.644585 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5074d440-0c80-45a6-8092-713f8c4283bc admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:24.673319 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 840/1712] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:24 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:24.674456 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5074d440-0c80-45a6-8092-713f8c4283bc admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:24.684919 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-5074d440-0c80-45a6-8092-713f8c4283bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:24.715842 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 873/1713] 10.222.0.40 () {58 vars in 1057 bytes} [Wed Jul 6 17:13:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:24.718231 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16131ff2-c259-4ebc-a3e9-2ddd3da0f3db None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:24.719239 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16131ff2-c259-4ebc-a3e9-2ddd3da0f3db None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:24.725169 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16131ff2-c259-4ebc-a3e9-2ddd3da0f3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:24.760650 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-16131ff2-c259-4ebc-a3e9-2ddd3da0f3db None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:24.762943 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-16131ff2-c259-4ebc-a3e9-2ddd3da0f3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:24.778875 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 874/1714] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:24.780624 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6612f-ae9c-49d2-ac92-ba13e370ba94 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:24.782160 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6612f-ae9c-49d2-ac92-ba13e370ba94 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:24.782656 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6612f-ae9c-49d2-ac92-ba13e370ba94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:24.805267 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-f7a6612f-ae9c-49d2-ac92-ba13e370ba94 None None] MFA Rules not processed for user `5f479458a68b4daf9f3a7ae53d6c3763`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:24.806733 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f7a6612f-ae9c-49d2-ac92-ba13e370ba94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:24.852272 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5074d440-0c80-45a6-8092-713f8c4283bc admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:24.895996 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea037f7-61ea-4831-8048-f781b0a44c18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:24.897070 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea037f7-61ea-4831-8048-f781b0a44c18 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:24.897363 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea037f7-61ea-4831-8048-f781b0a44c18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:24.897669 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea037f7-61ea-4831-8048-f781b0a44c18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:24.916317 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 875/1715] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 6 17:13:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:24.918391 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a6612f-ae9c-49d2-ac92-ba13e370ba94 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:24.929789 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:24.937418 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea037f7-61ea-4831-8048-f781b0a44c18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:24.945403 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea037f7-61ea-4831-8048-f781b0a44c18 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:24.954879 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a6612f-ae9c-49d2-ac92-ba13e370ba94 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:24.970249 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c37ed-cc25-49fd-b6eb-ab0ab1b2c857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:24.970249 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c37ed-cc25-49fd-b6eb-ab0ab1b2c857 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:24.970249 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c37ed-cc25-49fd-b6eb-ab0ab1b2c857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:24.970249 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c37ed-cc25-49fd-b6eb-ab0ab1b2c857 admin admin] PATH_INFO: `/v3/projects/e7c5b49b06f74faa976be579fbf7520f/users/6fbe6af960524d968d62bf5a5a0f3152/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:24.970641 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 841/1716] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:13:24.971914 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea037f7-61ea-4831-8048-f781b0a44c18 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:25.013722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea037f7-61ea-4831-8048-f781b0a44c18 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:25.022442 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37c37ed-cc25-49fd-b6eb-ab0ab1b2c857 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7c5b49b06f74faa976be579fbf7520f, user_id=6fbe6af960524d968d62bf5a5a0f3152, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:25.028411 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69671db-9d7f-4d80-8b58-c5be17934194 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76ca877bee9041d28ad41e45ef3dd18c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:25.029252 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69671db-9d7f-4d80-8b58-c5be17934194 service placement] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:25.033733 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37c37ed-cc25-49fd-b6eb-ab0ab1b2c857 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:25.034147 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69671db-9d7f-4d80-8b58-c5be17934194 service placement] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:25.034721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69671db-9d7f-4d80-8b58-c5be17934194 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:25.041075 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-f69671db-9d7f-4d80-8b58-c5be17934194 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:25.061458 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69671db-9d7f-4d80-8b58-c5be17934194 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:25.063376 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69671db-9d7f-4d80-8b58-c5be17934194 service placement] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:25.075018 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 842/1717] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:13:25.076173 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69671db-9d7f-4d80-8b58-c5be17934194 service placement] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:25.100957 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69671db-9d7f-4d80-8b58-c5be17934194 service placement] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:25.126075 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 876/1718] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:24 2022] PUT /identity/v3/projects/e7c5b49b06f74faa976be579fbf7520f/users/6fbe6af960524d968d62bf5a5a0f3152/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:25.131988 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c37ed-cc25-49fd-b6eb-ab0ab1b2c857 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:25.132294 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1053ff45-8ccc-4eb4-8edf-ae64976e3853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:25.132294 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053ff45-8ccc-4eb4-8edf-ae64976e3853 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:25.132294 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053ff45-8ccc-4eb4-8edf-ae64976e3853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:25.132294 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053ff45-8ccc-4eb4-8edf-ae64976e3853 admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:25.132620 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1053ff45-8ccc-4eb4-8edf-ae64976e3853 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:13:25.132620 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1053ff45-8ccc-4eb4-8edf-ae64976e3853 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:25.132767 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d37c37ed-cc25-49fd-b6eb-ab0ab1b2c857 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:25.132854 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1053ff45-8ccc-4eb4-8edf-ae64976e3853 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:25.184144 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 843/1719] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:13:25.186357 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1053ff45-8ccc-4eb4-8edf-ae64976e3853 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:25.221943 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1053ff45-8ccc-4eb4-8edf-ae64976e3853 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:25.323900 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:25.338442 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c37ed-cc25-49fd-b6eb-ab0ab1b2c857 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:25.361324 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe30749-aef2-402c-b128-f1c5b25ef819 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:25.369624 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe30749-aef2-402c-b128-f1c5b25ef819 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:25.369624 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe30749-aef2-402c-b128-f1c5b25ef819 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:25.369624 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe30749-aef2-402c-b128-f1c5b25ef819 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:25.379052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe30749-aef2-402c-b128-f1c5b25ef819 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:25.380748 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe30749-aef2-402c-b128-f1c5b25ef819 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:25.400455 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 877/1720] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:13:25.401869 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe30749-aef2-402c-b128-f1c5b25ef819 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:25.404434 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-1fe30749-aef2-402c-b128-f1c5b25ef819 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:25.405391 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b2e191-3c93-420a-8095-5e9dc1ff3b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:25.406673 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2e191-3c93-420a-8095-5e9dc1ff3b58 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:25.406963 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2e191-3c93-420a-8095-5e9dc1ff3b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:25.407271 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b2e191-3c93-420a-8095-5e9dc1ff3b58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:25.409342 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b2e191-3c93-420a-8095-5e9dc1ff3b58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:25.411011 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b2e191-3c93-420a-8095-5e9dc1ff3b58 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:25.434735 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 844/1721] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:25 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:25.435603 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b2e191-3c93-420a-8095-5e9dc1ff3b58 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:25.445002 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe30749-aef2-402c-b128-f1c5b25ef819 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:25.457360 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7e7f33-8ebd-4c4a-a18f-1bb512b19547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:25.459308 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7e7f33-8ebd-4c4a-a18f-1bb512b19547 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:25.459840 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7e7f33-8ebd-4c4a-a18f-1bb512b19547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:25.460268 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7e7f33-8ebd-4c4a-a18f-1bb512b19547 admin admin] PATH_INFO: `/v3/users/cc18962150634d0ead8afdb9c3f85074` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:25.464102 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7e7f33-8ebd-4c4a-a18f-1bb512b19547 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc18962150634d0ead8afdb9c3f85074)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:25.466368 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b2e191-3c93-420a-8095-5e9dc1ff3b58 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:25.468388 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7e7f33-8ebd-4c4a-a18f-1bb512b19547 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:25.472626 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085821ae-cf85-498e-b637-5e72323ec264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:25.473311 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085821ae-cf85-498e-b637-5e72323ec264 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:25.473596 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085821ae-cf85-498e-b637-5e72323ec264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:25.473875 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085821ae-cf85-498e-b637-5e72323ec264 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:25.474582 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085821ae-cf85-498e-b637-5e72323ec264 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:25.476691 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085821ae-cf85-498e-b637-5e72323ec264 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:25.630026 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 845/1722] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:13:25 2022] POST /identity/v3/projects => generated 387 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:25.631167 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085821ae-cf85-498e-b637-5e72323ec264 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:25.784852 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-af7e7f33-8ebd-4c4a-a18f-1bb512b19547 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc18962150634d0ead8afdb9c3f85074'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:25.810560 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-af7e7f33-8ebd-4c4a-a18f-1bb512b19547 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc18962150634d0ead8afdb9c3f85074'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:25.811349 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-af7e7f33-8ebd-4c4a-a18f-1bb512b19547 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc18962150634d0ead8afdb9c3f85074'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:25.821791 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085821ae-cf85-498e-b637-5e72323ec264 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:25.835375 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d864d6c5-5962-4c93-a26a-fad3eb2ed2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:25.837614 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864d6c5-5962-4c93-a26a-fad3eb2ed2e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:25.838020 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864d6c5-5962-4c93-a26a-fad3eb2ed2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:25.838475 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864d6c5-5962-4c93-a26a-fad3eb2ed2e1 admin admin] PATH_INFO: `/v3/projects/e7c5b49b06f74faa976be579fbf7520f/users/6fbe6af960524d968d62bf5a5a0f3152/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:25.841234 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d864d6c5-5962-4c93-a26a-fad3eb2ed2e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7c5b49b06f74faa976be579fbf7520f, user_id=6fbe6af960524d968d62bf5a5a0f3152, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:25.847587 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-af7e7f33-8ebd-4c4a-a18f-1bb512b19547 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc18962150634d0ead8afdb9c3f85074'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:25.900448 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d864d6c5-5962-4c93-a26a-fad3eb2ed2e1 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:25.939063 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 846/1723] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:25 2022] PUT /identity/v3/projects/e7c5b49b06f74faa976be579fbf7520f/users/6fbe6af960524d968d62bf5a5a0f3152/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:25.939063 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d864d6c5-5962-4c93-a26a-fad3eb2ed2e1 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:26.005286 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-af7e7f33-8ebd-4c4a-a18f-1bb512b19547 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc18962150634d0ead8afdb9c3f85074'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:26.015521 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 878/1724] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:13:25 2022] DELETE /identity/v3/users/cc18962150634d0ead8afdb9c3f85074 => generated 0 bytes in 611 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:26.017222 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2018a0c2-c651-456c-b13d-b05cfd0bc8b0 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:26.017518 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2018a0c2-c651-456c-b13d-b05cfd0bc8b0 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:26.017803 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2018a0c2-c651-456c-b13d-b05cfd0bc8b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:26.073108 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-2018a0c2-c651-456c-b13d-b05cfd0bc8b0 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:26.073108 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-2018a0c2-c651-456c-b13d-b05cfd0bc8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:26.093406 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d864d6c5-5962-4c93-a26a-fad3eb2ed2e1 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:26.271621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72021a-da7a-4a2d-a9ae-807b524e2a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:26.272919 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72021a-da7a-4a2d-a9ae-807b524e2a3b admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:26.273201 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72021a-da7a-4a2d-a9ae-807b524e2a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:26.273478 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72021a-da7a-4a2d-a9ae-807b524e2a3b admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:26.280185 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 879/1725] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:26.285578 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e72021a-da7a-4a2d-a9ae-807b524e2a3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:26.289379 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e72021a-da7a-4a2d-a9ae-807b524e2a3b admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:26.327492 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2018a0c2-c651-456c-b13d-b05cfd0bc8b0 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:26.331565 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-2018a0c2-c651-456c-b13d-b05cfd0bc8b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:26.413318 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 847/1726] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:13:25 2022] POST /identity/v3/users => generated 294 bytes in 477 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:26.418232 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72021a-da7a-4a2d-a9ae-807b524e2a3b admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:26.421462 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4e72021a-da7a-4a2d-a9ae-807b524e2a3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:26.507994 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72021a-da7a-4a2d-a9ae-807b524e2a3b admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:26.521164 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2018a0c2-c651-456c-b13d-b05cfd0bc8b0 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:26.526987 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d454833-a3ef-49c8-9a73-8644c076b96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:26.527694 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d454833-a3ef-49c8-9a73-8644c076b96e admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:26.530448 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d454833-a3ef-49c8-9a73-8644c076b96e admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:26.531270 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d454833-a3ef-49c8-9a73-8644c076b96e admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:26.534752 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fcd358-3a4f-4a32-8fe9-83ca8eac7e79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:26.535442 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcd358-3a4f-4a32-8fe9-83ca8eac7e79 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:26.535740 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcd358-3a4f-4a32-8fe9-83ca8eac7e79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:26.536026 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcd358-3a4f-4a32-8fe9-83ca8eac7e79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:26.538991 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d454833-a3ef-49c8-9a73-8644c076b96e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:26.544147 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d454833-a3ef-49c8-9a73-8644c076b96e admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:26.577659 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fcd358-3a4f-4a32-8fe9-83ca8eac7e79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:26.581816 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 848/1727] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:26.588953 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fcd358-3a4f-4a32-8fe9-83ca8eac7e79 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:26.591173 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d454833-a3ef-49c8-9a73-8644c076b96e admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:26.620255 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d454833-a3ef-49c8-9a73-8644c076b96e admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:26.631218 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99365ddd-fd2c-4d59-8b67-9384b34d32d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:26.631960 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99365ddd-fd2c-4d59-8b67-9384b34d32d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:26.632501 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99365ddd-fd2c-4d59-8b67-9384b34d32d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:26.632770 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99365ddd-fd2c-4d59-8b67-9384b34d32d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:26.633324 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 880/1728] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 17:13:26.645621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99365ddd-fd2c-4d59-8b67-9384b34d32d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:26.653700 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99365ddd-fd2c-4d59-8b67-9384b34d32d4 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:26.705886 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 849/1729] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:26 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:26.708724 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fcd358-3a4f-4a32-8fe9-83ca8eac7e79 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:26.751065 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fcd358-3a4f-4a32-8fe9-83ca8eac7e79 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:26.753195 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dedc81d-f722-4b2c-b529-505c0d5c2ef9 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:26.753195 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dedc81d-f722-4b2c-b529-505c0d5c2ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:26.753195 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dedc81d-f722-4b2c-b529-505c0d5c2ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:26.763288 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a2a6b6-9d4d-4679-9fb5-e908d7a40217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:26.764379 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2a6b6-9d4d-4679-9fb5-e908d7a40217 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:26.764929 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2a6b6-9d4d-4679-9fb5-e908d7a40217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:26.765943 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a2a6b6-9d4d-4679-9fb5-e908d7a40217 admin admin] PATH_INFO: `/v3/projects/b7f052cc276747b9bc9acbf22ccbfa1f/users/d88a987ae16e466c88478eef0bcb96e9/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:26.806372 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a2a6b6-9d4d-4679-9fb5-e908d7a40217 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7f052cc276747b9bc9acbf22ccbfa1f, user_id=d88a987ae16e466c88478eef0bcb96e9, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:26.806769 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-6dedc81d-f722-4b2c-b529-505c0d5c2ef9 None None] MFA Rules not processed for user `7922db6ad0ce41868a68f5a834a3b8f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:26.808763 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-6dedc81d-f722-4b2c-b529-505c0d5c2ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:26.809095 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a2a6b6-9d4d-4679-9fb5-e908d7a40217 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:26.939462 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 881/1730] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:26 2022] PUT /identity/v3/projects/b7f052cc276747b9bc9acbf22ccbfa1f/users/d88a987ae16e466c88478eef0bcb96e9/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:26.941912 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc08a8-0bb7-4f0d-ae19-702274947689 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:26.941912 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc08a8-0bb7-4f0d-ae19-702274947689 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:26.942093 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc08a8-0bb7-4f0d-ae19-702274947689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:26.961575 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 850/1731] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:26 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:26.971636 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:27.034122 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-02dc08a8-0bb7-4f0d-ae19-702274947689 None None] MFA Rules not processed for user `d88a987ae16e466c88478eef0bcb96e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:27.035422 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-02dc08a8-0bb7-4f0d-ae19-702274947689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:27.152564 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dedc81d-f722-4b2c-b529-505c0d5c2ef9 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:27.154051 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-6dedc81d-f722-4b2c-b529-505c0d5c2ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:27.182550 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 882/1732] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:27.184193 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc08a8-0bb7-4f0d-ae19-702274947689 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:27.193788 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dedc81d-f722-4b2c-b529-505c0d5c2ef9 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:27.264561 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dc08a8-0bb7-4f0d-ae19-702274947689 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:27.272142 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135b31a4-632b-4fb2-8f8e-130d2380e4d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:27.273104 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b31a4-632b-4fb2-8f8e-130d2380e4d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:27.273613 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b31a4-632b-4fb2-8f8e-130d2380e4d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:27.274149 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b31a4-632b-4fb2-8f8e-130d2380e4d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:27.275067 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135b31a4-632b-4fb2-8f8e-130d2380e4d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:27.278506 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135b31a4-632b-4fb2-8f8e-130d2380e4d1 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:27.423549 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308541c8-9e1d-47b8-915a-6192181d6502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:27.426818 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308541c8-9e1d-47b8-915a-6192181d6502 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:27.432099 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308541c8-9e1d-47b8-915a-6192181d6502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:27.432099 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308541c8-9e1d-47b8-915a-6192181d6502 admin admin] PATH_INFO: `/v3/projects/58ff8d20419140388a4e0baf04f66ddb` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:27.434883 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308541c8-9e1d-47b8-915a-6192181d6502 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58ff8d20419140388a4e0baf04f66ddb)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:27.438278 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308541c8-9e1d-47b8-915a-6192181d6502 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:27.444311 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 883/1733] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:13:27 2022] POST /identity/v3/projects => generated 385 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:27.449656 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135b31a4-632b-4fb2-8f8e-130d2380e4d1 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:27.520450 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:27.684001 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135b31a4-632b-4fb2-8f8e-130d2380e4d1 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:27.704531 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-308541c8-9e1d-47b8-915a-6192181d6502 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58ff8d20419140388a4e0baf04f66ddb', 'request_id': 'req-308541c8-9e1d-47b8-915a-6192181d6502'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:27.752166 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:27.752933 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-308541c8-9e1d-47b8-915a-6192181d6502 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58ff8d20419140388a4e0baf04f66ddb', 'request_id': 'req-308541c8-9e1d-47b8-915a-6192181d6502'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:27.868159 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e049dc-0d58-442b-9e06-35d522029963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:27.869330 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e049dc-0d58-442b-9e06-35d522029963 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:27.871388 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e049dc-0d58-442b-9e06-35d522029963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:27.871850 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e049dc-0d58-442b-9e06-35d522029963 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:27.874065 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e049dc-0d58-442b-9e06-35d522029963 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:27.878340 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e049dc-0d58-442b-9e06-35d522029963 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:27.886887 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-308541c8-9e1d-47b8-915a-6192181d6502 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58ff8d20419140388a4e0baf04f66ddb', 'request_id': 'req-308541c8-9e1d-47b8-915a-6192181d6502'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:27.931914 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-308541c8-9e1d-47b8-915a-6192181d6502 admin admin] The token cache is being invalidate because project 58ff8d20419140388a4e0baf04f66ddb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:13:27.932217 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-308541c8-9e1d-47b8-915a-6192181d6502 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:27.935721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-308541c8-9e1d-47b8-915a-6192181d6502 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:27.945312 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 851/1734] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:13:27 2022] DELETE /identity/v3/projects/58ff8d20419140388a4e0baf04f66ddb => generated 0 bytes in 789 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:13:27.956887 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308541c8-9e1d-47b8-915a-6192181d6502 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:27.960348 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-308541c8-9e1d-47b8-915a-6192181d6502 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:28.000234 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 884/1735] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:13:27 2022] POST /identity/v3/users => generated 293 bytes in 551 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:28.005130 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5241ba-9cfe-4693-a76e-268a20c49bda None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:28.005602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5241ba-9cfe-4693-a76e-268a20c49bda None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:28.008161 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5241ba-9cfe-4693-a76e-268a20c49bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:28.015841 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308541c8-9e1d-47b8-915a-6192181d6502 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:28.033411 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ab8a95-3282-4217-9a3f-88dfbcfb29a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:28.034555 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab8a95-3282-4217-9a3f-88dfbcfb29a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:28.035000 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab8a95-3282-4217-9a3f-88dfbcfb29a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:28.035430 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ab8a95-3282-4217-9a3f-88dfbcfb29a6 admin admin] PATH_INFO: `/v3/projects/42690acd6bce45ad9690e99f29f4552e` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:28.038330 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ab8a95-3282-4217-9a3f-88dfbcfb29a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42690acd6bce45ad9690e99f29f4552e)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:28.040517 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ab8a95-3282-4217-9a3f-88dfbcfb29a6 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:28.051853 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-1d5241ba-9cfe-4693-a76e-268a20c49bda None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:28.057650 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-1d5241ba-9cfe-4693-a76e-268a20c49bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:28.074369 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 885/1736] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:28.081504 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5241ba-9cfe-4693-a76e-268a20c49bda None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:28.081504 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-1d5241ba-9cfe-4693-a76e-268a20c49bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:28.105485 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5241ba-9cfe-4693-a76e-268a20c49bda None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:28.125110 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb19a48-be3b-4d51-95b7-4941465e636f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:28.127893 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb19a48-be3b-4d51-95b7-4941465e636f admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:28.128368 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb19a48-be3b-4d51-95b7-4941465e636f admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:28.128849 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb19a48-be3b-4d51-95b7-4941465e636f admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:28.129776 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb19a48-be3b-4d51-95b7-4941465e636f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:28.131769 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb19a48-be3b-4d51-95b7-4941465e636f admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:28.144542 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 886/1737] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:28.154618 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf02e0f-659b-4e21-8734-9163f212d2b7 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:28.154618 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf02e0f-659b-4e21-8734-9163f212d2b7 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:28.154618 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf02e0f-659b-4e21-8734-9163f212d2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:28.200939 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-bdf02e0f-659b-4e21-8734-9163f212d2b7 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:28.202188 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-bdf02e0f-659b-4e21-8734-9163f212d2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:28.235289 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 887/1738] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:28.236328 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf02e0f-659b-4e21-8734-9163f212d2b7 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:28.258513 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-95ab8a95-3282-4217-9a3f-88dfbcfb29a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42690acd6bce45ad9690e99f29f4552e', 'request_id': 'req-95ab8a95-3282-4217-9a3f-88dfbcfb29a6'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:28.262699 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf02e0f-659b-4e21-8734-9163f212d2b7 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:28.273947 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-95ab8a95-3282-4217-9a3f-88dfbcfb29a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42690acd6bce45ad9690e99f29f4552e', 'request_id': 'req-95ab8a95-3282-4217-9a3f-88dfbcfb29a6'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:28.468826 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-95ab8a95-3282-4217-9a3f-88dfbcfb29a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42690acd6bce45ad9690e99f29f4552e', 'request_id': 'req-95ab8a95-3282-4217-9a3f-88dfbcfb29a6'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:28.497946 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-95ab8a95-3282-4217-9a3f-88dfbcfb29a6 admin admin] The token cache is being invalidate because project 42690acd6bce45ad9690e99f29f4552e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:13:28.498251 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-95ab8a95-3282-4217-9a3f-88dfbcfb29a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:28.499934 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-95ab8a95-3282-4217-9a3f-88dfbcfb29a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:28.507670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 852/1739] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:13:27 2022] DELETE /identity/v3/projects/42690acd6bce45ad9690e99f29f4552e => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:28.508732 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ab8a95-3282-4217-9a3f-88dfbcfb29a6 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:28.518658 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-95ab8a95-3282-4217-9a3f-88dfbcfb29a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:28.532610 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ffe442-4335-4cb2-a953-18f825cf66a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:28.533275 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffe442-4335-4cb2-a953-18f825cf66a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:28.533565 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffe442-4335-4cb2-a953-18f825cf66a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:28.533851 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffe442-4335-4cb2-a953-18f825cf66a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:28.538889 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ffe442-4335-4cb2-a953-18f825cf66a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:28.541873 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ffe442-4335-4cb2-a953-18f825cf66a0 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:28.559334 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ab8a95-3282-4217-9a3f-88dfbcfb29a6 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:28.599150 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 888/1740] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 360 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:28.601526 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0d761-c2f6-45cf-901d-0aa09f57a029 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:28.603351 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0d761-c2f6-45cf-901d-0aa09f57a029 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:28.605494 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0d761-c2f6-45cf-901d-0aa09f57a029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:28.608479 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aab037b-03b1-4224-8882-938e7802104b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:28.609941 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aab037b-03b1-4224-8882-938e7802104b admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:28.610592 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aab037b-03b1-4224-8882-938e7802104b admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:28.611015 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aab037b-03b1-4224-8882-938e7802104b admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:28.612724 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aab037b-03b1-4224-8882-938e7802104b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:13:28.620797 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aab037b-03b1-4224-8882-938e7802104b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:28.622168 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aab037b-03b1-4224-8882-938e7802104b admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:28.678770 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-5fd0d761-c2f6-45cf-901d-0aa09f57a029 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:28.679227 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 853/1741] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:13:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:13:28.680964 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-5fd0d761-c2f6-45cf-901d-0aa09f57a029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:28.684692 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aab037b-03b1-4224-8882-938e7802104b admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:28.693213 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 889/1742] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:28.694896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0d761-c2f6-45cf-901d-0aa09f57a029 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:28.701269 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-9aab037b-03b1-4224-8882-938e7802104b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:28.711193 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0d761-c2f6-45cf-901d-0aa09f57a029 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:28.726934 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aab037b-03b1-4224-8882-938e7802104b admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:28.733902 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5035f1a6-57d9-4112-8deb-b59cfcfe3002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:28.735519 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035f1a6-57d9-4112-8deb-b59cfcfe3002 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:28.737887 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035f1a6-57d9-4112-8deb-b59cfcfe3002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:28.738380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5035f1a6-57d9-4112-8deb-b59cfcfe3002 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:28.740805 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5035f1a6-57d9-4112-8deb-b59cfcfe3002 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:28.745784 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5035f1a6-57d9-4112-8deb-b59cfcfe3002 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:28.763250 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d0199b-a2af-41cd-806f-bc0ae9cc620e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:28.764590 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d0199b-a2af-41cd-806f-bc0ae9cc620e admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:28.764880 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d0199b-a2af-41cd-806f-bc0ae9cc620e admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:28.765198 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d0199b-a2af-41cd-806f-bc0ae9cc620e admin admin] PATH_INFO: `/v3/projects/973d3ba0ed7b40c4b80ca6f685ece74b/users/4747c69e14704d03a7a1dc51a342059e/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:28.794824 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d0199b-a2af-41cd-806f-bc0ae9cc620e admin admin] RBAC: Authorizing `identity:create_grant(project_id=973d3ba0ed7b40c4b80ca6f685ece74b, user_id=4747c69e14704d03a7a1dc51a342059e, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:28.798682 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d0199b-a2af-41cd-806f-bc0ae9cc620e admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:28.851746 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 890/1743] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:13:28 2022] POST /identity/v3/projects => generated 383 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:28.863245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e641757-9c7d-4cfb-b6b0-129f224e98ab None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:28.868846 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e641757-9c7d-4cfb-b6b0-129f224e98ab None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:28.869238 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e641757-9c7d-4cfb-b6b0-129f224e98ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:28.885207 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 854/1744] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:28 2022] PUT /identity/v3/projects/973d3ba0ed7b40c4b80ca6f685ece74b/users/4747c69e14704d03a7a1dc51a342059e/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:28.889614 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d0199b-a2af-41cd-806f-bc0ae9cc620e admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:28.935630 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-1e641757-9c7d-4cfb-b6b0-129f224e98ab None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:28.937586 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-1e641757-9c7d-4cfb-b6b0-129f224e98ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:29.044921 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d0199b-a2af-41cd-806f-bc0ae9cc620e admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:29.050892 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c69739-fa14-4981-aac8-526083bc084a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:29.051647 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c69739-fa14-4981-aac8-526083bc084a admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:29.052383 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c69739-fa14-4981-aac8-526083bc084a admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:29.052821 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c69739-fa14-4981-aac8-526083bc084a admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:29.059063 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c69739-fa14-4981-aac8-526083bc084a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:29.061446 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c69739-fa14-4981-aac8-526083bc084a admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:29.065648 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 891/1745] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:29.067085 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b913b-f701-4d86-b733-d76ef3e33914 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:29.067427 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b913b-f701-4d86-b733-d76ef3e33914 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:29.067713 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086b913b-f701-4d86-b733-d76ef3e33914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:29.124902 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:29.159555 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-086b913b-f701-4d86-b733-d76ef3e33914 None None] MFA Rules not processed for user `4747c69e14704d03a7a1dc51a342059e`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:29.162384 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-086b913b-f701-4d86-b733-d76ef3e33914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:29.192500 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 855/1746] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:13:28 2022] POST /identity/v3/users => generated 292 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:29.192500 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c69739-fa14-4981-aac8-526083bc084a admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:29.209863 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c69739-fa14-4981-aac8-526083bc084a admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:29.221174 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb952ffa-824e-4536-b4a4-54e272daa8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:29.222292 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb952ffa-824e-4536-b4a4-54e272daa8b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:29.222792 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb952ffa-824e-4536-b4a4-54e272daa8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:29.223270 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb952ffa-824e-4536-b4a4-54e272daa8b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:29.224265 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb952ffa-824e-4536-b4a4-54e272daa8b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:13:29.224803 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb952ffa-824e-4536-b4a4-54e272daa8b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:29.227676 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb952ffa-824e-4536-b4a4-54e272daa8b0 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:29.260538 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 856/1747] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:13:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:13:29.262258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37e3ee-9736-4b9c-9724-87165070300b None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:29.262764 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37e3ee-9736-4b9c-9724-87165070300b None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:29.263251 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37e3ee-9736-4b9c-9724-87165070300b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:29.317268 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-3f37e3ee-9736-4b9c-9724-87165070300b None None] MFA Rules not processed for user `6fbe6af960524d968d62bf5a5a0f3152`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:29.319040 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-3f37e3ee-9736-4b9c-9724-87165070300b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:29.333640 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 892/1748] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:29.334686 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b913b-f701-4d86-b733-d76ef3e33914 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:29.358625 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086b913b-f701-4d86-b733-d76ef3e33914 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:29.375229 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a98a858-91ea-4cfb-862f-e003b1a03724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:29.375229 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98a858-91ea-4cfb-862f-e003b1a03724 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:29.375229 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98a858-91ea-4cfb-862f-e003b1a03724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:29.375229 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98a858-91ea-4cfb-862f-e003b1a03724 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:29.375572 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a98a858-91ea-4cfb-862f-e003b1a03724 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:29.375572 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a98a858-91ea-4cfb-862f-e003b1a03724 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:29.397026 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 893/1749] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:29 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:29.397026 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a98a858-91ea-4cfb-862f-e003b1a03724 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:29.415190 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a98a858-91ea-4cfb-862f-e003b1a03724 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:29.423968 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 857/1750] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:29 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:29.425234 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37e3ee-9736-4b9c-9724-87165070300b None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:29.439113 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37e3ee-9736-4b9c-9724-87165070300b None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:29.444645 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a1c420-b564-4266-b223-2be97edd0235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:29.460451 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a1c420-b564-4266-b223-2be97edd0235 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:29.460609 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69107be6-1ef4-4da7-9d3e-f2e55b24f525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:29.460609 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69107be6-1ef4-4da7-9d3e-f2e55b24f525 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:29.460777 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a1c420-b564-4266-b223-2be97edd0235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:29.460845 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69107be6-1ef4-4da7-9d3e-f2e55b24f525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:29.460909 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a1c420-b564-4266-b223-2be97edd0235 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:29.460981 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69107be6-1ef4-4da7-9d3e-f2e55b24f525 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:29.461053 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a1c420-b564-4266-b223-2be97edd0235 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:29.461124 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69107be6-1ef4-4da7-9d3e-f2e55b24f525 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:29.461194 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a1c420-b564-4266-b223-2be97edd0235 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:29.461264 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69107be6-1ef4-4da7-9d3e-f2e55b24f525 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:29.602794 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 858/1751] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:13:29 2022] POST /identity/v3/projects => generated 387 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:29.603660 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a1c420-b564-4266-b223-2be97edd0235 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:29.608404 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 894/1752] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:13:29 2022] POST /identity/v3/projects => generated 385 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:29.608404 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69107be6-1ef4-4da7-9d3e-f2e55b24f525 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:29.770482 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69107be6-1ef4-4da7-9d3e-f2e55b24f525 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:29.786864 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93feea6a-1100-4195-8ffb-6e670a6b3372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:29.787827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93feea6a-1100-4195-8ffb-6e670a6b3372 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:29.793589 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93feea6a-1100-4195-8ffb-6e670a6b3372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:29.793589 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93feea6a-1100-4195-8ffb-6e670a6b3372 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:29.793589 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93feea6a-1100-4195-8ffb-6e670a6b3372 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:29.793589 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93feea6a-1100-4195-8ffb-6e670a6b3372 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:29.805217 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a1c420-b564-4266-b223-2be97edd0235 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:29.818286 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2472f551-bdd6-4451-9151-8e340c98b4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:29.818990 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472f551-bdd6-4451-9151-8e340c98b4df admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:29.820393 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472f551-bdd6-4451-9151-8e340c98b4df admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:29.820681 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472f551-bdd6-4451-9151-8e340c98b4df admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:29.821442 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2472f551-bdd6-4451-9151-8e340c98b4df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:29.823664 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2472f551-bdd6-4451-9151-8e340c98b4df admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:29.842546 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 859/1753] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:29 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:29.843518 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2472f551-bdd6-4451-9151-8e340c98b4df admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:29.866173 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2472f551-bdd6-4451-9151-8e340c98b4df admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:29.871625 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f769e94c-8989-4a1d-829a-cb8f65fb22f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:29.872597 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f769e94c-8989-4a1d-829a-cb8f65fb22f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:29.873043 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f769e94c-8989-4a1d-829a-cb8f65fb22f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:29.873479 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f769e94c-8989-4a1d-829a-cb8f65fb22f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:29.874865 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f769e94c-8989-4a1d-829a-cb8f65fb22f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:29.877112 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f769e94c-8989-4a1d-829a-cb8f65fb22f6 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:29.924391 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 895/1754] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:13:29 2022] POST /identity/v3/users => generated 300 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:29.925847 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93feea6a-1100-4195-8ffb-6e670a6b3372 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:29.963390 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93feea6a-1100-4195-8ffb-6e670a6b3372 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:29.986830 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3eb7cf-0e7d-4e0e-b154-a5257e408880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:29.987801 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3eb7cf-0e7d-4e0e-b154-a5257e408880 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:29.988098 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3eb7cf-0e7d-4e0e-b154-a5257e408880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:29.988375 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3eb7cf-0e7d-4e0e-b154-a5257e408880 admin admin] PATH_INFO: `/v3/projects/d1565e79ae634de0adc621de33ef96ab/users/b3f21c541f164c58b7e6688edf32b577/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:30.029552 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3eb7cf-0e7d-4e0e-b154-a5257e408880 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1565e79ae634de0adc621de33ef96ab, user_id=b3f21c541f164c58b7e6688edf32b577, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:30.033974 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3eb7cf-0e7d-4e0e-b154-a5257e408880 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:30.077817 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 860/1755] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:13:29 2022] POST /identity/v3/users => generated 293 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:30.080324 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f769e94c-8989-4a1d-829a-cb8f65fb22f6 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:30.123342 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 896/1756] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:29 2022] PUT /identity/v3/projects/d1565e79ae634de0adc621de33ef96ab/users/b3f21c541f164c58b7e6688edf32b577/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:30.132196 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3eb7cf-0e7d-4e0e-b154-a5257e408880 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:30.135935 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f769e94c-8989-4a1d-829a-cb8f65fb22f6 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:30.152887 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:30.218361 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379d6f0b-0e97-4348-8b7a-757867f68e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:30.218361 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d6f0b-0e97-4348-8b7a-757867f68e58 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:30.218361 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d6f0b-0e97-4348-8b7a-757867f68e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:30.218361 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d6f0b-0e97-4348-8b7a-757867f68e58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:30.218670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379d6f0b-0e97-4348-8b7a-757867f68e58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:30.222039 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379d6f0b-0e97-4348-8b7a-757867f68e58 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:30.247422 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 861/1757] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:30.249072 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e551d-2fad-4989-960e-1c8db22566df None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:30.253394 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e551d-2fad-4989-960e-1c8db22566df None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:30.253394 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e551d-2fad-4989-960e-1c8db22566df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:30.285553 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-435e551d-2fad-4989-960e-1c8db22566df None None] MFA Rules not processed for user `b3f21c541f164c58b7e6688edf32b577`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:30.287128 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-435e551d-2fad-4989-960e-1c8db22566df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:30.353617 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3eb7cf-0e7d-4e0e-b154-a5257e408880 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:30.370950 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebeea44-7a57-4deb-b71d-34df38e0f5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:30.371910 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebeea44-7a57-4deb-b71d-34df38e0f5a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:30.372270 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebeea44-7a57-4deb-b71d-34df38e0f5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:30.372627 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebeea44-7a57-4deb-b71d-34df38e0f5a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:30.373857 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebeea44-7a57-4deb-b71d-34df38e0f5a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:30.375675 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebeea44-7a57-4deb-b71d-34df38e0f5a9 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:30.402967 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 897/1758] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:30.403676 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebeea44-7a57-4deb-b71d-34df38e0f5a9 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:30.415249 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 862/1759] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:30 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:30.416355 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e551d-2fad-4989-960e-1c8db22566df None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:30.431780 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e551d-2fad-4989-960e-1c8db22566df None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:30.433626 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebeea44-7a57-4deb-b71d-34df38e0f5a9 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:30.444100 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e260e19-8324-42c4-b877-69c10a5c4d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:30.444893 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e260e19-8324-42c4-b877-69c10a5c4d56 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:30.445252 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e260e19-8324-42c4-b877-69c10a5c4d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:30.445649 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e260e19-8324-42c4-b877-69c10a5c4d56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:30.446720 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e260e19-8324-42c4-b877-69c10a5c4d56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:30.449878 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4885b4-8bb8-48d5-9778-0b0c49411d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:30.451112 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e260e19-8324-42c4-b877-69c10a5c4d56 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:30.451604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4885b4-8bb8-48d5-9778-0b0c49411d79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:30.452067 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4885b4-8bb8-48d5-9778-0b0c49411d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:30.453182 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4885b4-8bb8-48d5-9778-0b0c49411d79 admin admin] PATH_INFO: `/v3/projects/45e9dbd4612b4e209e89664c3862700e/users/2a0c77cb5a0d42088f281b1f790fe22c/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:30.483079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 863/1760] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:30.485347 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e260e19-8324-42c4-b877-69c10a5c4d56 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:30.497742 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-5e260e19-8324-42c4-b877-69c10a5c4d56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:30.514302 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4885b4-8bb8-48d5-9778-0b0c49411d79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45e9dbd4612b4e209e89664c3862700e, user_id=2a0c77cb5a0d42088f281b1f790fe22c, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:30.521956 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4885b4-8bb8-48d5-9778-0b0c49411d79 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:30.623058 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 898/1761] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:30 2022] PUT /identity/v3/projects/45e9dbd4612b4e209e89664c3862700e/users/2a0c77cb5a0d42088f281b1f790fe22c/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:30.624158 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4885b4-8bb8-48d5-9778-0b0c49411d79 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:30.646867 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:30.676183 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e260e19-8324-42c4-b877-69c10a5c4d56 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:30.817698 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8806af90-255f-4ad4-b346-3b5d4830409e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:30.818509 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8806af90-255f-4ad4-b346-3b5d4830409e service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:30.818805 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8806af90-255f-4ad4-b346-3b5d4830409e service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:30.819090 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8806af90-255f-4ad4-b346-3b5d4830409e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:30.836156 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4885b4-8bb8-48d5-9778-0b0c49411d79 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:30.841736 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f7b0e-57da-4093-856a-60ad90437c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:30.843434 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f7b0e-57da-4093-856a-60ad90437c55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:30.844051 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f7b0e-57da-4093-856a-60ad90437c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:30.844506 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504f7b0e-57da-4093-856a-60ad90437c55 admin admin] PATH_INFO: `/v3/projects/92bdb0c6c09d4242bd6ca394a44c9752/users/d6ed0d69605f41c7a937128b1511f194/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:30.865655 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f7b0e-57da-4093-856a-60ad90437c55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92bdb0c6c09d4242bd6ca394a44c9752, user_id=d6ed0d69605f41c7a937128b1511f194, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:30.869644 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504f7b0e-57da-4093-856a-60ad90437c55 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:30.967345 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8806af90-255f-4ad4-b346-3b5d4830409e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:30.969549 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8806af90-255f-4ad4-b346-3b5d4830409e service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:30.981979 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 899/1762] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:30 2022] PUT /identity/v3/projects/92bdb0c6c09d4242bd6ca394a44c9752/users/d6ed0d69605f41c7a937128b1511f194/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:30.981979 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f7b0e-57da-4093-856a-60ad90437c55 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:31.009058 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 864/1763] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:30 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 525 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:13:31.010462 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e45210-2a86-4089-9072-241c8b05e8f3 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:31.010756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e45210-2a86-4089-9072-241c8b05e8f3 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:31.011036 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e45210-2a86-4089-9072-241c8b05e8f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:31.116756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-41e45210-2a86-4089-9072-241c8b05e8f3 None None] MFA Rules not processed for user `d6ed0d69605f41c7a937128b1511f194`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:31.118282 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-41e45210-2a86-4089-9072-241c8b05e8f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:31.161096 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504f7b0e-57da-4093-856a-60ad90437c55 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:31.169265 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e863852-408f-45cd-a150-ef0463690a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:31.170049 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e863852-408f-45cd-a150-ef0463690a20 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:31.170337 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e863852-408f-45cd-a150-ef0463690a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:31.171923 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e863852-408f-45cd-a150-ef0463690a20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:31.177708 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e863852-408f-45cd-a150-ef0463690a20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:31.177708 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e863852-408f-45cd-a150-ef0463690a20 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:31.211698 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 900/1764] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:30 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:31.211698 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35598e71-ffef-4066-9cf4-70fd70692fea None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:31.211698 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35598e71-ffef-4066-9cf4-70fd70692fea None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:31.211698 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35598e71-ffef-4066-9cf4-70fd70692fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:31.262424 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-35598e71-ffef-4066-9cf4-70fd70692fea None None] MFA Rules not processed for user `1905e6f8c3e24e3cbfefce2827baa370`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:31.268150 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-35598e71-ffef-4066-9cf4-70fd70692fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:31.318456 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 865/1765] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:31.318456 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e45210-2a86-4089-9072-241c8b05e8f3 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:31.340901 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e45210-2a86-4089-9072-241c8b05e8f3 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:31.362109 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb90d9-be77-4570-beac-dca64de188d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:31.363135 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb90d9-be77-4570-beac-dca64de188d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:31.364385 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb90d9-be77-4570-beac-dca64de188d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:31.366047 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb90d9-be77-4570-beac-dca64de188d4 admin admin] PATH_INFO: `/v3/projects/45e9dbd4612b4e209e89664c3862700e/users/2a0c77cb5a0d42088f281b1f790fe22c/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:31.373679 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb90d9-be77-4570-beac-dca64de188d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45e9dbd4612b4e209e89664c3862700e, user_id=2a0c77cb5a0d42088f281b1f790fe22c, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:31.383430 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb90d9-be77-4570-beac-dca64de188d4 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:31.394860 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:31.428594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 866/1766] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:31 2022] PUT /identity/v3/projects/45e9dbd4612b4e209e89664c3862700e/users/2a0c77cb5a0d42088f281b1f790fe22c/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:31.428594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb90d9-be77-4570-beac-dca64de188d4 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:31.453817 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 901/1767] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 6 17:13:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:31.456383 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0bef3-50e9-4a9e-b390-c7c0e0426f58 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:31.457185 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0bef3-50e9-4a9e-b390-c7c0e0426f58 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:31.474400 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0bef3-50e9-4a9e-b390-c7c0e0426f58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:31.536095 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-dac0bef3-50e9-4a9e-b390-c7c0e0426f58 None None] MFA Rules not processed for user `2a0c77cb5a0d42088f281b1f790fe22c`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:31.537576 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-dac0bef3-50e9-4a9e-b390-c7c0e0426f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:31.587136 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb90d9-be77-4570-beac-dca64de188d4 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:31.600439 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f910c5-4440-4996-b5f7-56119d8ce70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:31.601020 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f910c5-4440-4996-b5f7-56119d8ce70e admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:31.601306 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f910c5-4440-4996-b5f7-56119d8ce70e admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:31.601586 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f910c5-4440-4996-b5f7-56119d8ce70e admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:31.602286 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f910c5-4440-4996-b5f7-56119d8ce70e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:31.604246 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f910c5-4440-4996-b5f7-56119d8ce70e admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:31.774671 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 867/1768] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:13:31 2022] POST /identity/v3/projects => generated 387 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:31.784968 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a3c94-0ddf-4e71-971e-8a69f3cdb9a8 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:31.785259 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a3c94-0ddf-4e71-971e-8a69f3cdb9a8 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:31.785560 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6a3c94-0ddf-4e71-971e-8a69f3cdb9a8 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:31.789547 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 868/1769] 10.222.0.40 () {54 vars in 921 bytes} [Wed Jul 6 17:13:31 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:13:31.790318 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a3c94-0ddf-4e71-971e-8a69f3cdb9a8 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:31.802665 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 902/1770] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:31 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:31.859147 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547836c2-ab05-4025-80c3-ca3c9aeb29e1 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:31.860101 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547836c2-ab05-4025-80c3-ca3c9aeb29e1 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:31.860539 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547836c2-ab05-4025-80c3-ca3c9aeb29e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:31.935985 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-547836c2-ab05-4025-80c3-ca3c9aeb29e1 None None] MFA Rules not processed for user `76ca877bee9041d28ad41e45ef3dd18c`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:31.940700 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-547836c2-ab05-4025-80c3-ca3c9aeb29e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:31.987565 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6a3c94-0ddf-4e71-971e-8a69f3cdb9a8 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:32.001597 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174974ef-ebdc-40f5-b86e-9eba0cbd5ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:32.002758 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174974ef-ebdc-40f5-b86e-9eba0cbd5ff0 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:32.004990 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174974ef-ebdc-40f5-b86e-9eba0cbd5ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:32.005289 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174974ef-ebdc-40f5-b86e-9eba0cbd5ff0 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:32.006019 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174974ef-ebdc-40f5-b86e-9eba0cbd5ff0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:32.009306 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174974ef-ebdc-40f5-b86e-9eba0cbd5ff0 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:32.099547 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 903/1771] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Jul 6 17:13:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:32.136175 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547836c2-ab05-4025-80c3-ca3c9aeb29e1 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:32.137938 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-547836c2-ab05-4025-80c3-ca3c9aeb29e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:32.159583 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 869/1772] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:13:31 2022] POST /identity/v3/users => generated 300 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:32.165328 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174974ef-ebdc-40f5-b86e-9eba0cbd5ff0 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:32.189401 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174974ef-ebdc-40f5-b86e-9eba0cbd5ff0 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:32.206060 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b6490-e230-4ed8-b41f-edc90685a931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:32.225081 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b6490-e230-4ed8-b41f-edc90685a931 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:32.225389 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b6490-e230-4ed8-b41f-edc90685a931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:32.225684 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20b6490-e230-4ed8-b41f-edc90685a931 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:32.226490 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20b6490-e230-4ed8-b41f-edc90685a931 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:32.237956 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20b6490-e230-4ed8-b41f-edc90685a931 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:32.282922 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 870/1773] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:32.293475 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b6490-e230-4ed8-b41f-edc90685a931 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:32.298243 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e20b6490-e230-4ed8-b41f-edc90685a931 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:32.339662 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20b6490-e230-4ed8-b41f-edc90685a931 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:32.362724 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb944237-b953-4578-aa04-d5fd4170be7b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76ca877bee9041d28ad41e45ef3dd18c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:32.364862 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb944237-b953-4578-aa04-d5fd4170be7b service placement] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:32.366634 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb944237-b953-4578-aa04-d5fd4170be7b service placement] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:32.366973 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb944237-b953-4578-aa04-d5fd4170be7b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:32.384125 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547836c2-ab05-4025-80c3-ca3c9aeb29e1 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:32.392825 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93e50b6-8e34-4deb-91b6-90f7f3cfd53c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75acacd30e9a4cf2ad58b072b49e3f05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:32.395664 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e50b6-8e34-4deb-91b6-90f7f3cfd53c service swift] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:32.396061 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e50b6-8e34-4deb-91b6-90f7f3cfd53c service swift] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:32.396422 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93e50b6-8e34-4deb-91b6-90f7f3cfd53c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:32.400612 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb944237-b953-4578-aa04-d5fd4170be7b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:32.403454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb944237-b953-4578-aa04-d5fd4170be7b service placement] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:32.440891 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 871/1774] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:13:32.444062 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb944237-b953-4578-aa04-d5fd4170be7b service placement] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:32.449387 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-fb944237-b953-4578-aa04-d5fd4170be7b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:32.602917 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93e50b6-8e34-4deb-91b6-90f7f3cfd53c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:32.610266 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93e50b6-8e34-4deb-91b6-90f7f3cfd53c service swift] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:32.622003 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb944237-b953-4578-aa04-d5fd4170be7b service placement] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:32.632552 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 904/1775] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 6 17:13:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 17:13:32.640653 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5130e6f5-d4ff-44b9-a3eb-fb83cca4534e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:32.648643 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93e50b6-8e34-4deb-91b6-90f7f3cfd53c service swift] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:32.667600 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:INSERT=5,keystone:DELETE=3,keystone:SELECT=92 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:32.670610 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130e6f5-d4ff-44b9-a3eb-fb83cca4534e service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:32.671196 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130e6f5-d4ff-44b9-a3eb-fb83cca4534e service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:32.671554 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5130e6f5-d4ff-44b9-a3eb-fb83cca4534e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:32.713629 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93e50b6-8e34-4deb-91b6-90f7f3cfd53c service swift] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:32.757928 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00c572-7a3b-42ee-a517-7e2344e63125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:32.760405 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00c572-7a3b-42ee-a517-7e2344e63125 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:32.765265 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00c572-7a3b-42ee-a517-7e2344e63125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:32.765265 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00c572-7a3b-42ee-a517-7e2344e63125 admin admin] PATH_INFO: `/v3/projects/4b2c3a71fd094b55955f75ffbe50feba/users/a257986b275d453f8933598b27c29c45/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:32.832541 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c00c572-7a3b-42ee-a517-7e2344e63125 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b2c3a71fd094b55955f75ffbe50feba, user_id=a257986b275d453f8933598b27c29c45, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:32.835239 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c00c572-7a3b-42ee-a517-7e2344e63125 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:32.852588 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5130e6f5-d4ff-44b9-a3eb-fb83cca4534e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:32.855096 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5130e6f5-d4ff-44b9-a3eb-fb83cca4534e service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:32.896801 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 872/1776] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:32 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:13:32.975840 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 905/1777] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:32 2022] PUT /identity/v3/projects/4b2c3a71fd094b55955f75ffbe50feba/users/a257986b275d453f8933598b27c29c45/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:32.981523 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5130e6f5-d4ff-44b9-a3eb-fb83cca4534e service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:33.013171 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5130e6f5-d4ff-44b9-a3eb-fb83cca4534e service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:33.290969 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48bf9e4-7510-444d-8788-10d27c208a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:33.291945 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bf9e4-7510-444d-8788-10d27c208a95 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:33.292471 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bf9e4-7510-444d-8788-10d27c208a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:33.292995 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48bf9e4-7510-444d-8788-10d27c208a95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:33.294213 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48bf9e4-7510-444d-8788-10d27c208a95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:33.298040 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48bf9e4-7510-444d-8788-10d27c208a95 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:33.324531 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00c572-7a3b-42ee-a517-7e2344e63125 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:33.340346 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00c572-7a3b-42ee-a517-7e2344e63125 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:33.344597 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 873/1778] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:33.374372 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ef012-a254-4dc2-8eaa-227a55622ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:33.378758 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ef012-a254-4dc2-8eaa-227a55622ff9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:33.378758 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ef012-a254-4dc2-8eaa-227a55622ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:33.378758 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ef012-a254-4dc2-8eaa-227a55622ff9 admin admin] PATH_INFO: `/v3/projects/4b2c3a71fd094b55955f75ffbe50feba/users/a257986b275d453f8933598b27c29c45/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:33.397258 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ef012-a254-4dc2-8eaa-227a55622ff9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b2c3a71fd094b55955f75ffbe50feba, user_id=a257986b275d453f8933598b27c29c45, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:33.407869 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ef012-a254-4dc2-8eaa-227a55622ff9 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:33.440675 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 906/1779] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:33 2022] PUT /identity/v3/projects/4b2c3a71fd094b55955f75ffbe50feba/users/a257986b275d453f8933598b27c29c45/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:33.466860 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995bcf2e-79dd-432a-936c-5cc24c19e7d6 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:33.467462 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995bcf2e-79dd-432a-936c-5cc24c19e7d6 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:33.468120 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995bcf2e-79dd-432a-936c-5cc24c19e7d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:33.533636 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-995bcf2e-79dd-432a-936c-5cc24c19e7d6 None None] MFA Rules not processed for user `a257986b275d453f8933598b27c29c45`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:33.535052 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-995bcf2e-79dd-432a-936c-5cc24c19e7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:33.707334 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:33.829516 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 874/1780] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:33 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:34.172237 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ef012-a254-4dc2-8eaa-227a55622ff9 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:34.301204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995bcf2e-79dd-432a-936c-5cc24c19e7d6 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:34.303815 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-995bcf2e-79dd-432a-936c-5cc24c19e7d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:34.306500 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ef012-a254-4dc2-8eaa-227a55622ff9 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:34.311181 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ac0a8-8d9e-454c-b410-b5226936ef94 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:34.313027 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ac0a8-8d9e-454c-b410-b5226936ef94 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:34.313660 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ac0a8-8d9e-454c-b410-b5226936ef94 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:34.314217 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ac0a8-8d9e-454c-b410-b5226936ef94 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:34.331437 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328ac0a8-8d9e-454c-b410-b5226936ef94 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:34.333829 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328ac0a8-8d9e-454c-b410-b5226936ef94 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:34.335409 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:34.373157 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 907/1781] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:34 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:13:34.507236 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6402ed7-0485-4e6a-9860-3128080411a1 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:34.509253 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6402ed7-0485-4e6a-9860-3128080411a1 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:34.509855 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6402ed7-0485-4e6a-9860-3128080411a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:34.634393 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-c6402ed7-0485-4e6a-9860-3128080411a1 None None] MFA Rules not processed for user `5f479458a68b4daf9f3a7ae53d6c3763`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:34.634393 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c6402ed7-0485-4e6a-9860-3128080411a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:34.635776 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995bcf2e-79dd-432a-936c-5cc24c19e7d6 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:34.635776 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc53e7-79c2-4d79-aee6-300a6ad5cd0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:34.635776 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc53e7-79c2-4d79-aee6-300a6ad5cd0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:34.635776 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc53e7-79c2-4d79-aee6-300a6ad5cd0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:34.636775 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbc53e7-79c2-4d79-aee6-300a6ad5cd0d admin admin] PATH_INFO: `/v3/users/a8c6e03fe8c14d5899eadc2b26e9c44c` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:34.636775 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbc53e7-79c2-4d79-aee6-300a6ad5cd0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8c6e03fe8c14d5899eadc2b26e9c44c)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:34.636775 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbc53e7-79c2-4d79-aee6-300a6ad5cd0d admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:34.705666 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 908/1782] 10.222.0.50 () {58 vars in 1015 bytes} [Wed Jul 6 17:13:34 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:34.753005 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6402ed7-0485-4e6a-9860-3128080411a1 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:34.754729 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c6402ed7-0485-4e6a-9860-3128080411a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:34.771571 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6402ed7-0485-4e6a-9860-3128080411a1 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:34.779280 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179dcaa3-5506-4016-b79d-44c26e0dea34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:34.779280 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179dcaa3-5506-4016-b79d-44c26e0dea34 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:34.781926 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179dcaa3-5506-4016-b79d-44c26e0dea34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:34.782410 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179dcaa3-5506-4016-b79d-44c26e0dea34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:34.802884 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:34.807032 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179dcaa3-5506-4016-b79d-44c26e0dea34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:34.808244 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179dcaa3-5506-4016-b79d-44c26e0dea34 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:34.822390 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 909/1783] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:13:34.951837 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-3fbc53e7-79c2-4d79-aee6-300a6ad5cd0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8c6e03fe8c14d5899eadc2b26e9c44c'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:35.001914 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-3fbc53e7-79c2-4d79-aee6-300a6ad5cd0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8c6e03fe8c14d5899eadc2b26e9c44c'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:35.013570 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-3fbc53e7-79c2-4d79-aee6-300a6ad5cd0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8c6e03fe8c14d5899eadc2b26e9c44c'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:35.048979 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-3fbc53e7-79c2-4d79-aee6-300a6ad5cd0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8c6e03fe8c14d5899eadc2b26e9c44c'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:35.266694 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-3fbc53e7-79c2-4d79-aee6-300a6ad5cd0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8c6e03fe8c14d5899eadc2b26e9c44c'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:35.277902 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 875/1784] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:13:34 2022] DELETE /identity/v3/users/a8c6e03fe8c14d5899eadc2b26e9c44c => generated 0 bytes in 978 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:35.412624 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179dcaa3-5506-4016-b79d-44c26e0dea34 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:35.428340 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179dcaa3-5506-4016-b79d-44c26e0dea34 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:35.598916 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c4474d-88fa-400b-9072-bcddbddb7055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:35.599857 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4474d-88fa-400b-9072-bcddbddb7055 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:35.600140 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4474d-88fa-400b-9072-bcddbddb7055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:35.600424 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4474d-88fa-400b-9072-bcddbddb7055 admin admin] PATH_INFO: `/v3/users/d6ed0d69605f41c7a937128b1511f194` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:35.606110 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c4474d-88fa-400b-9072-bcddbddb7055 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6ed0d69605f41c7a937128b1511f194)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:35.609057 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c4474d-88fa-400b-9072-bcddbddb7055 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:35.950828 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc53e7-79c2-4d79-aee6-300a6ad5cd0d admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:35.950828 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-3fbc53e7-79c2-4d79-aee6-300a6ad5cd0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:36.021623 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbc53e7-79c2-4d79-aee6-300a6ad5cd0d admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:36.026234 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf5b79-ff22-4a78-b4b7-38844edb477f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:36.027302 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf5b79-ff22-4a78-b4b7-38844edb477f service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:36.027654 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf5b79-ff22-4a78-b4b7-38844edb477f service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:36.029092 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbf5b79-ff22-4a78-b4b7-38844edb477f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:36.029977 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ffbf5b79-ff22-4a78-b4b7-38844edb477f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:36.101911 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:36.120845 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-18c4474d-88fa-400b-9072-bcddbddb7055 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6ed0d69605f41c7a937128b1511f194'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:36.136585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-18c4474d-88fa-400b-9072-bcddbddb7055 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6ed0d69605f41c7a937128b1511f194'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:36.140565 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-18c4474d-88fa-400b-9072-bcddbddb7055 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6ed0d69605f41c7a937128b1511f194'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:36.164114 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-18c4474d-88fa-400b-9072-bcddbddb7055 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6ed0d69605f41c7a937128b1511f194'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:36.169811 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf5b79-ff22-4a78-b4b7-38844edb477f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:36.173499 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbf5b79-ff22-4a78-b4b7-38844edb477f service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:36.196554 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 876/1785] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 17:13:36.196554 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf5b79-ff22-4a78-b4b7-38844edb477f service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:36.197400 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ffbf5b79-ff22-4a78-b4b7-38844edb477f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:36.216083 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbf5b79-ff22-4a78-b4b7-38844edb477f service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:36.221769 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54ccb26-a231-4d4c-b912-860dad2bfc1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:36.222772 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ccb26-a231-4d4c-b912-860dad2bfc1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:36.223133 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ccb26-a231-4d4c-b912-860dad2bfc1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:36.224410 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54ccb26-a231-4d4c-b912-860dad2bfc1a admin admin] PATH_INFO: `/v3/projects/e703c86373a64c1386a02f301eb1bbbf` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:36.226867 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-18c4474d-88fa-400b-9072-bcddbddb7055 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6ed0d69605f41c7a937128b1511f194'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:36.238490 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 910/1786] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:13:35 2022] DELETE /identity/v3/users/d6ed0d69605f41c7a937128b1511f194 => generated 0 bytes in 828 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:13:36.238646 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54ccb26-a231-4d4c-b912-860dad2bfc1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e703c86373a64c1386a02f301eb1bbbf)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:36.240033 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3833713-798d-42a5-af91-2dfc764c895a None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:36.240331 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3833713-798d-42a5-af91-2dfc764c895a None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:36.240780 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3833713-798d-42a5-af91-2dfc764c895a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:36.241330 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54ccb26-a231-4d4c-b912-860dad2bfc1a admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:36.263369 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-b3833713-798d-42a5-af91-2dfc764c895a None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:36.265777 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b3833713-798d-42a5-af91-2dfc764c895a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:36.351367 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 911/1787] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:13:36.376505 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:36.406996 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3833713-798d-42a5-af91-2dfc764c895a None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:36.408655 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b3833713-798d-42a5-af91-2dfc764c895a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:36.412449 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-e54ccb26-a231-4d4c-b912-860dad2bfc1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e703c86373a64c1386a02f301eb1bbbf', 'request_id': 'req-e54ccb26-a231-4d4c-b912-860dad2bfc1a'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:36.432370 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-e54ccb26-a231-4d4c-b912-860dad2bfc1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e703c86373a64c1386a02f301eb1bbbf', 'request_id': 'req-e54ccb26-a231-4d4c-b912-860dad2bfc1a'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:36.494214 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-e54ccb26-a231-4d4c-b912-860dad2bfc1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e703c86373a64c1386a02f301eb1bbbf', 'request_id': 'req-e54ccb26-a231-4d4c-b912-860dad2bfc1a'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:36.517659 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-e54ccb26-a231-4d4c-b912-860dad2bfc1a admin admin] The token cache is being invalidate because project e703c86373a64c1386a02f301eb1bbbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:13:36.518211 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-e54ccb26-a231-4d4c-b912-860dad2bfc1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:36.522007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-e54ccb26-a231-4d4c-b912-860dad2bfc1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:36.530313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 877/1788] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:13:36 2022] DELETE /identity/v3/projects/e703c86373a64c1386a02f301eb1bbbf => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:36.534503 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54ccb26-a231-4d4c-b912-860dad2bfc1a admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:36.537118 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e54ccb26-a231-4d4c-b912-860dad2bfc1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:36.548134 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3833713-798d-42a5-af91-2dfc764c895a None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:36.640564 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3c9287-30e5-4396-8cf6-a1cf7b7d8b83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:36.641405 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c9287-30e5-4396-8cf6-a1cf7b7d8b83 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:36.641718 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c9287-30e5-4396-8cf6-a1cf7b7d8b83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:36.641999 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3c9287-30e5-4396-8cf6-a1cf7b7d8b83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:36.644041 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54ccb26-a231-4d4c-b912-860dad2bfc1a admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:36.644172 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-0a3c9287-30e5-4396-8cf6-a1cf7b7d8b83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:36.650401 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49069ef0-8ca6-48b9-8e71-a324b5fe2692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:36.651746 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49069ef0-8ca6-48b9-8e71-a324b5fe2692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:36.652155 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49069ef0-8ca6-48b9-8e71-a324b5fe2692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:36.652536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49069ef0-8ca6-48b9-8e71-a324b5fe2692 admin admin] PATH_INFO: `/v3/users/6fbe6af960524d968d62bf5a5a0f3152` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:36.654763 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49069ef0-8ca6-48b9-8e71-a324b5fe2692 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fbe6af960524d968d62bf5a5a0f3152)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:36.658132 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49069ef0-8ca6-48b9-8e71-a324b5fe2692 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:36.663110 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3c9287-30e5-4396-8cf6-a1cf7b7d8b83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:36.666272 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3c9287-30e5-4396-8cf6-a1cf7b7d8b83 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:36.711001 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 912/1789] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:13:36.843488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:36.872114 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30422d-d0db-4823-a084-ce54ca0f7ad8 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:36.872406 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30422d-d0db-4823-a084-ce54ca0f7ad8 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:36.872685 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30422d-d0db-4823-a084-ce54ca0f7ad8 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:36.874426 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 913/1790] 10.222.0.40 () {54 vars in 921 bytes} [Wed Jul 6 17:13:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 06 17:13:36.887970 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24ce63-6f85-452d-9587-904f7c3275f9 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:36.888260 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24ce63-6f85-452d-9587-904f7c3275f9 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:36.888541 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24ce63-6f85-452d-9587-904f7c3275f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:36.930625 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-df24ce63-6f85-452d-9587-904f7c3275f9 None None] MFA Rules not processed for user `af26ef96dc9d47dc91549b736939268e`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:36.933729 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-df24ce63-6f85-452d-9587-904f7c3275f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:36.962440 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-49069ef0-8ca6-48b9-8e71-a324b5fe2692 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fbe6af960524d968d62bf5a5a0f3152'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:36.993773 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-49069ef0-8ca6-48b9-8e71-a324b5fe2692 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fbe6af960524d968d62bf5a5a0f3152'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:37.002474 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-49069ef0-8ca6-48b9-8e71-a324b5fe2692 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fbe6af960524d968d62bf5a5a0f3152'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:37.003419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-49069ef0-8ca6-48b9-8e71-a324b5fe2692 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fbe6af960524d968d62bf5a5a0f3152'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:37.044587 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 914/1791] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Jul 6 17:13:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:13:37.068239 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df24ce63-6f85-452d-9587-904f7c3275f9 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:37.069835 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-df24ce63-6f85-452d-9587-904f7c3275f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:37.085591 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df24ce63-6f85-452d-9587-904f7c3275f9 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:37.089962 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-49069ef0-8ca6-48b9-8e71-a324b5fe2692 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fbe6af960524d968d62bf5a5a0f3152'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:37.101427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 878/1792] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:13:36 2022] DELETE /identity/v3/users/6fbe6af960524d968d62bf5a5a0f3152 => generated 0 bytes in 567 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:13:37.104833 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18101cc-7d59-44f1-82a3-e4599f2b7201 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:37.105681 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18101cc-7d59-44f1-82a3-e4599f2b7201 service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:37.105895 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18101cc-7d59-44f1-82a3-e4599f2b7201 service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:37.106182 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18101cc-7d59-44f1-82a3-e4599f2b7201 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:37.109634 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b18101cc-7d59-44f1-82a3-e4599f2b7201 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:37.118494 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af7759-95fd-4e28-a674-2beddc909b49 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:37.119020 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af7759-95fd-4e28-a674-2beddc909b49 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:37.119852 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40af7759-95fd-4e28-a674-2beddc909b49 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:37.129979 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18101cc-7d59-44f1-82a3-e4599f2b7201 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:37.131606 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18101cc-7d59-44f1-82a3-e4599f2b7201 service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:37.137108 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 879/1793] 10.222.0.40 () {54 vars in 921 bytes} [Wed Jul 6 17:13:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:13:37.138619 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e975a6c-ff28-4797-b3d8-c27d0a534a0e None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:37.138915 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e975a6c-ff28-4797-b3d8-c27d0a534a0e None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:37.145338 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e975a6c-ff28-4797-b3d8-c27d0a534a0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:37.178389 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 915/1794] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 17:13:37.221510 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-8e975a6c-ff28-4797-b3d8-c27d0a534a0e None None] MFA Rules not processed for user `af26ef96dc9d47dc91549b736939268e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:37.224137 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-8e975a6c-ff28-4797-b3d8-c27d0a534a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:37.371087 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 880/1795] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Jul 6 17:13:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:37.480192 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18101cc-7d59-44f1-82a3-e4599f2b7201 service nova] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:37.481831 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b18101cc-7d59-44f1-82a3-e4599f2b7201 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:37.496241 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18101cc-7d59-44f1-82a3-e4599f2b7201 service nova] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:37.502023 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e67249e-1415-407f-aa3d-8fdb759d9772 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:37.502579 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67249e-1415-407f-aa3d-8fdb759d9772 service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:37.502870 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67249e-1415-407f-aa3d-8fdb759d9772 service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:37.503153 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67249e-1415-407f-aa3d-8fdb759d9772 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:37.513123 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e67249e-1415-407f-aa3d-8fdb759d9772 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:37.515981 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e67249e-1415-407f-aa3d-8fdb759d9772 service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:37.539108 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 916/1796] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:13:37.582407 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e975a6c-ff28-4797-b3d8-c27d0a534a0e None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:37.585035 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-8e975a6c-ff28-4797-b3d8-c27d0a534a0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:37.639050 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e67249e-1415-407f-aa3d-8fdb759d9772 service nova] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:37.644432 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-3e67249e-1415-407f-aa3d-8fdb759d9772 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:37.666906 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e975a6c-ff28-4797-b3d8-c27d0a534a0e None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:37.672402 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79031c0a-120b-40c3-87b5-cfc7398ffc9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:37.673277 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79031c0a-120b-40c3-87b5-cfc7398ffc9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:37.673570 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79031c0a-120b-40c3-87b5-cfc7398ffc9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:37.673850 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79031c0a-120b-40c3-87b5-cfc7398ffc9b admin admin] PATH_INFO: `/v3/projects/92bdb0c6c09d4242bd6ca394a44c9752` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:37.675062 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79031c0a-120b-40c3-87b5-cfc7398ffc9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=92bdb0c6c09d4242bd6ca394a44c9752)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:37.677190 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79031c0a-120b-40c3-87b5-cfc7398ffc9b admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:37.721155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e67249e-1415-407f-aa3d-8fdb759d9772 service nova] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:37.722177 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4cd43f-7719-4f6a-b787-242d16dcdbc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:37.723046 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4cd43f-7719-4f6a-b787-242d16dcdbc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:37.723332 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4cd43f-7719-4f6a-b787-242d16dcdbc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:37.723615 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4cd43f-7719-4f6a-b787-242d16dcdbc5 admin admin] PATH_INFO: `/v3/projects/e7c5b49b06f74faa976be579fbf7520f` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:37.724855 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4cd43f-7719-4f6a-b787-242d16dcdbc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7c5b49b06f74faa976be579fbf7520f)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:37.726943 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4cd43f-7719-4f6a-b787-242d16dcdbc5 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:37.791567 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-79031c0a-120b-40c3-87b5-cfc7398ffc9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92bdb0c6c09d4242bd6ca394a44c9752', 'request_id': 'req-79031c0a-120b-40c3-87b5-cfc7398ffc9b'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:37.797692 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-79031c0a-120b-40c3-87b5-cfc7398ffc9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92bdb0c6c09d4242bd6ca394a44c9752', 'request_id': 'req-79031c0a-120b-40c3-87b5-cfc7398ffc9b'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:37.826487 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:37.871684 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7a4cd43f-7719-4f6a-b787-242d16dcdbc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7c5b49b06f74faa976be579fbf7520f', 'request_id': 'req-7a4cd43f-7719-4f6a-b787-242d16dcdbc5'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:37.877884 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7a4cd43f-7719-4f6a-b787-242d16dcdbc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7c5b49b06f74faa976be579fbf7520f', 'request_id': 'req-7a4cd43f-7719-4f6a-b787-242d16dcdbc5'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:37.883789 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7a4cd43f-7719-4f6a-b787-242d16dcdbc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7c5b49b06f74faa976be579fbf7520f', 'request_id': 'req-7a4cd43f-7719-4f6a-b787-242d16dcdbc5'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:37.894773 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-79031c0a-120b-40c3-87b5-cfc7398ffc9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92bdb0c6c09d4242bd6ca394a44c9752', 'request_id': 'req-79031c0a-120b-40c3-87b5-cfc7398ffc9b'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:37.906990 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-79031c0a-120b-40c3-87b5-cfc7398ffc9b admin admin] The token cache is being invalidate because project 92bdb0c6c09d4242bd6ca394a44c9752 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:13:37.907286 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-79031c0a-120b-40c3-87b5-cfc7398ffc9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:37.907968 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-79031c0a-120b-40c3-87b5-cfc7398ffc9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:37.909523 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 881/1797] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:13:37 2022] DELETE /identity/v3/projects/92bdb0c6c09d4242bd6ca394a44c9752 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:37.913493 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79031c0a-120b-40c3-87b5-cfc7398ffc9b admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:37.914815 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-79031c0a-120b-40c3-87b5-cfc7398ffc9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:37.979190 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79031c0a-120b-40c3-87b5-cfc7398ffc9b admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:37.996679 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7a4cd43f-7719-4f6a-b787-242d16dcdbc5 admin admin] The token cache is being invalidate because project e7c5b49b06f74faa976be579fbf7520f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:13:37.996969 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7a4cd43f-7719-4f6a-b787-242d16dcdbc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:37.997880 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7a4cd43f-7719-4f6a-b787-242d16dcdbc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:37.999708 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 917/1798] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:13:37 2022] DELETE /identity/v3/projects/e7c5b49b06f74faa976be579fbf7520f => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:13:38.004275 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf691b-a924-4f7e-9d3d-5cfcbd00bfe9 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:38.004415 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf691b-a924-4f7e-9d3d-5cfcbd00bfe9 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:38.004689 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bf691b-a924-4f7e-9d3d-5cfcbd00bfe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:38.024309 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-88bf691b-a924-4f7e-9d3d-5cfcbd00bfe9 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:38.032008 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-88bf691b-a924-4f7e-9d3d-5cfcbd00bfe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:38.044614 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a040f3de-93f3-4b1d-8ae4-b47a39df5dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:38.049839 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a040f3de-93f3-4b1d-8ae4-b47a39df5dfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:38.050225 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a040f3de-93f3-4b1d-8ae4-b47a39df5dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:38.050576 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a040f3de-93f3-4b1d-8ae4-b47a39df5dfd admin admin] PATH_INFO: `/v3/users/a257986b275d453f8933598b27c29c45` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:38.054066 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a040f3de-93f3-4b1d-8ae4-b47a39df5dfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=a257986b275d453f8933598b27c29c45)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:38.061061 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a040f3de-93f3-4b1d-8ae4-b47a39df5dfd admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:38.086028 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:38.131898 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 918/1799] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:13:38.137603 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a8a00-b7c9-47f2-844a-c6539578043c None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:38.137833 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a8a00-b7c9-47f2-844a-c6539578043c None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:38.138122 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6a8a00-b7c9-47f2-844a-c6539578043c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:38.157848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-3d6a8a00-b7c9-47f2-844a-c6539578043c None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:38.159092 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-3d6a8a00-b7c9-47f2-844a-c6539578043c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:38.178805 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 919/1800] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:38.180406 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a8a00-b7c9-47f2-844a-c6539578043c None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:38.192206 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6a8a00-b7c9-47f2-844a-c6539578043c None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:38.200938 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad8567-b1eb-46a2-ae76-e5a4bec24849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:38.200938 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad8567-b1eb-46a2-ae76-e5a4bec24849 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:38.200938 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad8567-b1eb-46a2-ae76-e5a4bec24849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:38.200938 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffad8567-b1eb-46a2-ae76-e5a4bec24849 admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:38.201331 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffad8567-b1eb-46a2-ae76-e5a4bec24849 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:13:38.201331 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffad8567-b1eb-46a2-ae76-e5a4bec24849 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:38.202554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffad8567-b1eb-46a2-ae76-e5a4bec24849 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:38.214838 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 920/1801] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:13:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 17:13:38.220046 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad8567-b1eb-46a2-ae76-e5a4bec24849 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:38.230574 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffad8567-b1eb-46a2-ae76-e5a4bec24849 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:38.243133 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbc9c58-b081-4df5-a695-efcd4bf7a239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:38.244330 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc9c58-b081-4df5-a695-efcd4bf7a239 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:38.244909 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc9c58-b081-4df5-a695-efcd4bf7a239 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:38.245410 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbc9c58-b081-4df5-a695-efcd4bf7a239 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:38.246408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbc9c58-b081-4df5-a695-efcd4bf7a239 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:38.249799 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbc9c58-b081-4df5-a695-efcd4bf7a239 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:38.271483 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-a040f3de-93f3-4b1d-8ae4-b47a39df5dfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a257986b275d453f8933598b27c29c45'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:38.286929 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-a040f3de-93f3-4b1d-8ae4-b47a39df5dfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a257986b275d453f8933598b27c29c45'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:38.294390 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-a040f3de-93f3-4b1d-8ae4-b47a39df5dfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a257986b275d453f8933598b27c29c45'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:38.295336 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-a040f3de-93f3-4b1d-8ae4-b47a39df5dfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a257986b275d453f8933598b27c29c45'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:38.668039 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-a040f3de-93f3-4b1d-8ae4-b47a39df5dfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a257986b275d453f8933598b27c29c45'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:38.695522 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 882/1802] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:13:37 2022] DELETE /identity/v3/users/a257986b275d453f8933598b27c29c45 => generated 0 bytes in 768 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:13:38.696231 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 921/1803] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:13:38 2022] POST /identity/v3/projects => generated 363 bytes in 477 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:38.710577 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a040f3de-93f3-4b1d-8ae4-b47a39df5dfd admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:38.714460 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-a040f3de-93f3-4b1d-8ae4-b47a39df5dfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:38.843985 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a040f3de-93f3-4b1d-8ae4-b47a39df5dfd admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:38.903981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563105fa-52b6-4919-b38a-cbe22da07fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:38.905081 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563105fa-52b6-4919-b38a-cbe22da07fcf admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:38.905081 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563105fa-52b6-4919-b38a-cbe22da07fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:38.905081 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563105fa-52b6-4919-b38a-cbe22da07fcf admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:38.905674 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563105fa-52b6-4919-b38a-cbe22da07fcf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:38.908074 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563105fa-52b6-4919-b38a-cbe22da07fcf admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:39.049035 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 883/1804] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:13:38 2022] POST /identity/v3/users => generated 282 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:39.052742 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbc9c58-b081-4df5-a695-efcd4bf7a239 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:39.074087 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbc9c58-b081-4df5-a695-efcd4bf7a239 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:39.084740 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ab8ff8-6366-4c7a-920b-067e10b9c087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:39.086021 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab8ff8-6366-4c7a-920b-067e10b9c087 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:39.086532 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab8ff8-6366-4c7a-920b-067e10b9c087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:39.088547 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ab8ff8-6366-4c7a-920b-067e10b9c087 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:39.089410 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ab8ff8-6366-4c7a-920b-067e10b9c087 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:39.092000 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ab8ff8-6366-4c7a-920b-067e10b9c087 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:39.114889 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 922/1805] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:39.116924 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563105fa-52b6-4919-b38a-cbe22da07fcf admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:39.135795 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563105fa-52b6-4919-b38a-cbe22da07fcf admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:39.144389 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42f5b00-c349-42b4-ab12-7098c92aef7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:39.145080 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f5b00-c349-42b4-ab12-7098c92aef7f admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:39.145553 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f5b00-c349-42b4-ab12-7098c92aef7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:39.145915 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42f5b00-c349-42b4-ab12-7098c92aef7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:39.148009 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42f5b00-c349-42b4-ab12-7098c92aef7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:39.151174 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42f5b00-c349-42b4-ab12-7098c92aef7f admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:39.166497 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 884/1806] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:39 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:39.176205 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ab8ff8-6366-4c7a-920b-067e10b9c087 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:39.193654 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ab8ff8-6366-4c7a-920b-067e10b9c087 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:39.200097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0847495c-3f58-44f5-9169-9945a377bd90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:39.201662 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847495c-3f58-44f5-9169-9945a377bd90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:39.201662 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847495c-3f58-44f5-9169-9945a377bd90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:39.201894 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0847495c-3f58-44f5-9169-9945a377bd90 admin admin] PATH_INFO: `/v3/projects/16cdde3692144dc08a3d3915d903db02/users/c34dfe8ae9fc4c268bfb2f627a43b046/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:39.238592 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0847495c-3f58-44f5-9169-9945a377bd90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16cdde3692144dc08a3d3915d903db02, user_id=c34dfe8ae9fc4c268bfb2f627a43b046, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:39.240502 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:39.253008 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0847495c-3f58-44f5-9169-9945a377bd90 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:39.702514 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 923/1807] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:39 2022] PUT /identity/v3/projects/16cdde3692144dc08a3d3915d903db02/users/c34dfe8ae9fc4c268bfb2f627a43b046/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:39.704863 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c3681d-7e8a-433f-a4c5-aa4ac7314ac2 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:39.705131 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c3681d-7e8a-433f-a4c5-aa4ac7314ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:39.705383 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c3681d-7e8a-433f-a4c5-aa4ac7314ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:39.760274 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-65c3681d-7e8a-433f-a4c5-aa4ac7314ac2 None None] MFA Rules not processed for user `c34dfe8ae9fc4c268bfb2f627a43b046`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:39.761530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-65c3681d-7e8a-433f-a4c5-aa4ac7314ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:39.840590 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0847495c-3f58-44f5-9169-9945a377bd90 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:39.843405 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-0847495c-3f58-44f5-9169-9945a377bd90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:39.876664 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 885/1808] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:39 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:39.878651 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c3681d-7e8a-433f-a4c5-aa4ac7314ac2 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:40.066629 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0847495c-3f58-44f5-9169-9945a377bd90 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:40.074645 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f87cc52-1baa-4790-ba54-e686c3dd15a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:40.075635 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87cc52-1baa-4790-ba54-e686c3dd15a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:40.076344 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87cc52-1baa-4790-ba54-e686c3dd15a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:40.076344 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f87cc52-1baa-4790-ba54-e686c3dd15a6 admin admin] PATH_INFO: `/v3/projects/4b2c3a71fd094b55955f75ffbe50feba` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:40.080074 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f87cc52-1baa-4790-ba54-e686c3dd15a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b2c3a71fd094b55955f75ffbe50feba)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:40.081775 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f87cc52-1baa-4790-ba54-e686c3dd15a6 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:40.083605 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c3681d-7e8a-433f-a4c5-aa4ac7314ac2 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:40.090031 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbebe1f-e344-4a21-9940-7adaf4e22f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:40.091155 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbebe1f-e344-4a21-9940-7adaf4e22f7e admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:40.091769 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbebe1f-e344-4a21-9940-7adaf4e22f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:40.093254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbebe1f-e344-4a21-9940-7adaf4e22f7e admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:40.094418 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbebe1f-e344-4a21-9940-7adaf4e22f7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:40.097878 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbebe1f-e344-4a21-9940-7adaf4e22f7e admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:40.486801 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 886/1809] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:13:39 2022] POST /identity/v3/projects => generated 361 bytes in 609 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:40.488432 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbebe1f-e344-4a21-9940-7adaf4e22f7e admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:40.494210 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8f87cc52-1baa-4790-ba54-e686c3dd15a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b2c3a71fd094b55955f75ffbe50feba', 'request_id': 'req-8f87cc52-1baa-4790-ba54-e686c3dd15a6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:40.506096 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8f87cc52-1baa-4790-ba54-e686c3dd15a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b2c3a71fd094b55955f75ffbe50feba', 'request_id': 'req-8f87cc52-1baa-4790-ba54-e686c3dd15a6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:40.515416 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8f87cc52-1baa-4790-ba54-e686c3dd15a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b2c3a71fd094b55955f75ffbe50feba', 'request_id': 'req-8f87cc52-1baa-4790-ba54-e686c3dd15a6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:40.610516 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:40.610948 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbebe1f-e344-4a21-9940-7adaf4e22f7e admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:40.626949 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d2c94-126e-4216-a4df-4d097bdc355f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:40.627817 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d2c94-126e-4216-a4df-4d097bdc355f admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:40.636184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d2c94-126e-4216-a4df-4d097bdc355f admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:40.636184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d2c94-126e-4216-a4df-4d097bdc355f admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:40.636184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93d2c94-126e-4216-a4df-4d097bdc355f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:40.639070 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93d2c94-126e-4216-a4df-4d097bdc355f admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:40.672046 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8f87cc52-1baa-4790-ba54-e686c3dd15a6 admin admin] The token cache is being invalidate because project 4b2c3a71fd094b55955f75ffbe50feba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:13:40.803337 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8f87cc52-1baa-4790-ba54-e686c3dd15a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:40.803337 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8f87cc52-1baa-4790-ba54-e686c3dd15a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:40.803337 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 924/1810] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:13:39 2022] DELETE /identity/v3/projects/4b2c3a71fd094b55955f75ffbe50feba => generated 0 bytes in 836 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:40.803337 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99182f-5be4-40b8-858d-0b989ba36bb5 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:40.803337 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99182f-5be4-40b8-858d-0b989ba36bb5 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:40.803337 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99182f-5be4-40b8-858d-0b989ba36bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:40.856481 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-2a99182f-5be4-40b8-858d-0b989ba36bb5 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:40.865040 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-2a99182f-5be4-40b8-858d-0b989ba36bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:40.893878 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 925/1811] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:13:40.896191 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed4e21-99be-4706-98aa-13815f9c6b70 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:40.896488 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed4e21-99be-4706-98aa-13815f9c6b70 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:40.896753 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed4e21-99be-4706-98aa-13815f9c6b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:40.953588 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-eeed4e21-99be-4706-98aa-13815f9c6b70 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:40.955479 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-eeed4e21-99be-4706-98aa-13815f9c6b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:40.959918 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 887/1812] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:13:40 2022] POST /identity/v3/users => generated 287 bytes in 472 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:40.961157 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d2c94-126e-4216-a4df-4d097bdc355f admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:40.971406 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 926/1813] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:13:40.977275 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d93d2c94-126e-4216-a4df-4d097bdc355f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:40.984553 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed4e21-99be-4706-98aa-13815f9c6b70 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:40.987667 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d2c94-126e-4216-a4df-4d097bdc355f admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:40.993645 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b3e349-2f68-4b05-8e37-79293adf1776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:40.994338 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3e349-2f68-4b05-8e37-79293adf1776 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:40.994752 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3e349-2f68-4b05-8e37-79293adf1776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:40.995223 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b3e349-2f68-4b05-8e37-79293adf1776 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:40.998135 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b3e349-2f68-4b05-8e37-79293adf1776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:41.000225 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b3e349-2f68-4b05-8e37-79293adf1776 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:41.001130 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed4e21-99be-4706-98aa-13815f9c6b70 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:41.007750 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcbab7e-f8d5-420a-9a00-918a29df5aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:41.007750 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcbab7e-f8d5-420a-9a00-918a29df5aac admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:41.007750 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcbab7e-f8d5-420a-9a00-918a29df5aac admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:41.007998 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcbab7e-f8d5-420a-9a00-918a29df5aac admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:41.008895 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcbab7e-f8d5-420a-9a00-918a29df5aac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:13:41.009192 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcbab7e-f8d5-420a-9a00-918a29df5aac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:41.010780 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcbab7e-f8d5-420a-9a00-918a29df5aac admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:41.016605 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 888/1814] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:40 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:41.017689 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b3e349-2f68-4b05-8e37-79293adf1776 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:41.032488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b3e349-2f68-4b05-8e37-79293adf1776 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:41.040979 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc7d9ad-2c4b-4e46-8518-bca4575256eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:41.042091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc7d9ad-2c4b-4e46-8518-bca4575256eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:41.042457 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc7d9ad-2c4b-4e46-8518-bca4575256eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:41.044007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc7d9ad-2c4b-4e46-8518-bca4575256eb admin admin] PATH_INFO: `/v3/projects/d7d7dbfdc0774456b89d2d17b4f74f7e/users/915a1b3393bf4384a2d4d9a2bb8559e8/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:41.055460 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 927/1815] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:13:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:13:41.059779 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcbab7e-f8d5-420a-9a00-918a29df5aac admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:41.088152 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcbab7e-f8d5-420a-9a00-918a29df5aac admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:41.096055 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc7d9ad-2c4b-4e46-8518-bca4575256eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7d7dbfdc0774456b89d2d17b4f74f7e, user_id=915a1b3393bf4384a2d4d9a2bb8559e8, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:41.099542 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc7d9ad-2c4b-4e46-8518-bca4575256eb admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:41.100511 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70e1a3-2537-4b9f-937c-a17fc0ec033c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:41.101425 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70e1a3-2537-4b9f-937c-a17fc0ec033c admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:41.102309 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70e1a3-2537-4b9f-937c-a17fc0ec033c admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:41.102582 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70e1a3-2537-4b9f-937c-a17fc0ec033c admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:41.103311 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c70e1a3-2537-4b9f-937c-a17fc0ec033c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:41.109205 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c70e1a3-2537-4b9f-937c-a17fc0ec033c admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:41.291301 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 889/1816] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:41 2022] PUT /identity/v3/projects/d7d7dbfdc0774456b89d2d17b4f74f7e/users/915a1b3393bf4384a2d4d9a2bb8559e8/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:41.292275 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc7d9ad-2c4b-4e46-8518-bca4575256eb admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:41.299264 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 928/1817] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:13:41 2022] POST /identity/v3/projects => generated 387 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:41.305650 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70e1a3-2537-4b9f-937c-a17fc0ec033c admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:41.622832 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70e1a3-2537-4b9f-937c-a17fc0ec033c admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:41.624894 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc7d9ad-2c4b-4e46-8518-bca4575256eb admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:41.625281 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e2c018-315b-4bdd-9ef5-692668b87dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:41.625281 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e2c018-315b-4bdd-9ef5-692668b87dc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:41.625281 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e2c018-315b-4bdd-9ef5-692668b87dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:41.625281 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e2c018-315b-4bdd-9ef5-692668b87dc2 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:41.625563 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e2c018-315b-4bdd-9ef5-692668b87dc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:41.625563 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e2c018-315b-4bdd-9ef5-692668b87dc2 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:41.625692 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b13157-d24e-425d-bd6a-6c85b46f080d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:41.625692 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b13157-d24e-425d-bd6a-6c85b46f080d admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:41.625692 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b13157-d24e-425d-bd6a-6c85b46f080d admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:41.625692 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b13157-d24e-425d-bd6a-6c85b46f080d admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:41.625953 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b13157-d24e-425d-bd6a-6c85b46f080d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:41.625953 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b13157-d24e-425d-bd6a-6c85b46f080d admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:41.625953 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 890/1818] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:41.625953 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b13157-d24e-425d-bd6a-6c85b46f080d admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:41.625953 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b13157-d24e-425d-bd6a-6c85b46f080d admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:41.628733 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5466426-866b-4842-bd6e-51c4ef3202e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:41.629851 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5466426-866b-4842-bd6e-51c4ef3202e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:41.630085 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5466426-866b-4842-bd6e-51c4ef3202e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:41.630383 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5466426-866b-4842-bd6e-51c4ef3202e5 admin admin] PATH_INFO: `/v3/projects/d7d7dbfdc0774456b89d2d17b4f74f7e/users/915a1b3393bf4384a2d4d9a2bb8559e8/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:41.633987 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5466426-866b-4842-bd6e-51c4ef3202e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7d7dbfdc0774456b89d2d17b4f74f7e, user_id=915a1b3393bf4384a2d4d9a2bb8559e8, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:41.636378 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5466426-866b-4842-bd6e-51c4ef3202e5 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:41.655894 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 891/1819] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:41 2022] PUT /identity/v3/projects/d7d7dbfdc0774456b89d2d17b4f74f7e/users/915a1b3393bf4384a2d4d9a2bb8559e8/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:41.666496 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7821463-8c6f-49f4-87ab-db3b01dbb022 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:41.666817 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7821463-8c6f-49f4-87ab-db3b01dbb022 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:41.667125 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7821463-8c6f-49f4-87ab-db3b01dbb022 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:41.743244 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-b7821463-8c6f-49f4-87ab-db3b01dbb022 None None] MFA Rules not processed for user `915a1b3393bf4384a2d4d9a2bb8559e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:41.743845 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-b7821463-8c6f-49f4-87ab-db3b01dbb022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:41.840655 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 929/1820] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:13:41 2022] POST /identity/v3/users => generated 294 bytes in 530 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:41.842324 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e2c018-315b-4bdd-9ef5-692668b87dc2 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:41.872805 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e2c018-315b-4bdd-9ef5-692668b87dc2 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:41.941540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 892/1821] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:41 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:42.000395 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7821463-8c6f-49f4-87ab-db3b01dbb022 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:42.004006 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-b7821463-8c6f-49f4-87ab-db3b01dbb022 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:42.061698 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ec12a-0b03-4380-9e6c-c6f68e533970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:42.064345 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ec12a-0b03-4380-9e6c-c6f68e533970 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:42.066413 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ec12a-0b03-4380-9e6c-c6f68e533970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:42.078880 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232ec12a-0b03-4380-9e6c-c6f68e533970 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:42.083454 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ec12a-0b03-4380-9e6c-c6f68e533970 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:42.088205 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232ec12a-0b03-4380-9e6c-c6f68e533970 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:42.150857 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 930/1822] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:41 2022] GET /identity/v3/roles => generated 1420 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:13:42.162035 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ec12a-0b03-4380-9e6c-c6f68e533970 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:42.196221 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232ec12a-0b03-4380-9e6c-c6f68e533970 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:42.226604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace39c0-01f1-4973-9411-56705f187e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:42.226604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace39c0-01f1-4973-9411-56705f187e4c admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:42.226604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace39c0-01f1-4973-9411-56705f187e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:42.226604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dace39c0-01f1-4973-9411-56705f187e4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:42.226994 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dace39c0-01f1-4973-9411-56705f187e4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:42.227693 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7821463-8c6f-49f4-87ab-db3b01dbb022 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:42.232548 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dace39c0-01f1-4973-9411-56705f187e4c admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:42.234791 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66082edb-bcf1-4497-9442-2ed9bb23c76b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:42.235472 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66082edb-bcf1-4497-9442-2ed9bb23c76b service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:42.235995 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66082edb-bcf1-4497-9442-2ed9bb23c76b service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:42.236352 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66082edb-bcf1-4497-9442-2ed9bb23c76b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:42.259433 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66082edb-bcf1-4497-9442-2ed9bb23c76b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:42.261054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66082edb-bcf1-4497-9442-2ed9bb23c76b service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:42.273647 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:42.274499 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 931/1823] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:42.283035 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace39c0-01f1-4973-9411-56705f187e4c admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:42.292013 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 893/1824] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:41 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:13:42.304563 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dace39c0-01f1-4973-9411-56705f187e4c admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:42.323296 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b5f71b-5152-4ae8-9054-af2fe4cd06f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:42.325421 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5f71b-5152-4ae8-9054-af2fe4cd06f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:42.325778 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5f71b-5152-4ae8-9054-af2fe4cd06f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:42.326063 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b5f71b-5152-4ae8-9054-af2fe4cd06f0 admin admin] PATH_INFO: `/v3/projects/085e5740a6c84bb6918d9a3fe93a8852/users/eb472b67fcf54a4ebe1be96a7d1bd4d4/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:42.375330 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b5f71b-5152-4ae8-9054-af2fe4cd06f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=085e5740a6c84bb6918d9a3fe93a8852, user_id=eb472b67fcf54a4ebe1be96a7d1bd4d4, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:42.378664 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b5f71b-5152-4ae8-9054-af2fe4cd06f0 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:42.677728 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 932/1825] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:42 2022] PUT /identity/v3/projects/085e5740a6c84bb6918d9a3fe93a8852/users/eb472b67fcf54a4ebe1be96a7d1bd4d4/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:42.690170 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b606b-08f8-4f00-b20b-d91e849ed0e9 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:42.690566 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b606b-08f8-4f00-b20b-d91e849ed0e9 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:42.691048 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9b606b-08f8-4f00-b20b-d91e849ed0e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:42.827191 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-0d9b606b-08f8-4f00-b20b-d91e849ed0e9 None None] MFA Rules not processed for user `eb472b67fcf54a4ebe1be96a7d1bd4d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:42.833341 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-0d9b606b-08f8-4f00-b20b-d91e849ed0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:43.029967 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b5f71b-5152-4ae8-9054-af2fe4cd06f0 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:43.031843 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 894/1826] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:43.047320 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9b606b-08f8-4f00-b20b-d91e849ed0e9 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:43.050750 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-0d9b606b-08f8-4f00-b20b-d91e849ed0e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:43.130455 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b5f71b-5152-4ae8-9054-af2fe4cd06f0 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:43.138586 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cc4f86-b281-4a4c-93ab-f2cbc8a13867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:43.138586 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc4f86-b281-4a4c-93ab-f2cbc8a13867 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:43.139046 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc4f86-b281-4a4c-93ab-f2cbc8a13867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:43.140000 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cc4f86-b281-4a4c-93ab-f2cbc8a13867 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:43.153859 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9b606b-08f8-4f00-b20b-d91e849ed0e9 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:43.160235 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d1f3c-1910-4fbc-babb-f2ec9d5e877f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:43.161103 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d1f3c-1910-4fbc-babb-f2ec9d5e877f service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:43.161608 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d1f3c-1910-4fbc-babb-f2ec9d5e877f service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:43.162040 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d1f3c-1910-4fbc-babb-f2ec9d5e877f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:43.164242 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cc4f86-b281-4a4c-93ab-f2cbc8a13867 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:43.173239 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-0f3d1f3c-1910-4fbc-babb-f2ec9d5e877f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:43.178722 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cc4f86-b281-4a4c-93ab-f2cbc8a13867 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:43.332466 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 933/1827] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:13:43 2022] POST /identity/v3/projects => generated 385 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:13:43.333439 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cc4f86-b281-4a4c-93ab-f2cbc8a13867 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:43.335662 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c1cc4f86-b281-4a4c-93ab-f2cbc8a13867 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:43.363164 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3d1f3c-1910-4fbc-babb-f2ec9d5e877f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:43.364553 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3d1f3c-1910-4fbc-babb-f2ec9d5e877f service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:43.391529 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 895/1828] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:43 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 17:13:43.392602 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d1f3c-1910-4fbc-babb-f2ec9d5e877f service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:43.405308 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d1f3c-1910-4fbc-babb-f2ec9d5e877f service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:43.515205 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cc4f86-b281-4a4c-93ab-f2cbc8a13867 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:43.520672 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f9e4e8-c500-4c9e-a8bc-c4f4801e47cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:43.521490 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f9e4e8-c500-4c9e-a8bc-c4f4801e47cf admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:43.521876 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f9e4e8-c500-4c9e-a8bc-c4f4801e47cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:43.522255 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f9e4e8-c500-4c9e-a8bc-c4f4801e47cf admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:43.523056 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f9e4e8-c500-4c9e-a8bc-c4f4801e47cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:43.530237 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7cd789-e541-4c00-8f4a-7f4dbcfdecaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:43.531155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7cd789-e541-4c00-8f4a-7f4dbcfdecaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:43.531446 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7cd789-e541-4c00-8f4a-7f4dbcfdecaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:43.531758 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7cd789-e541-4c00-8f4a-7f4dbcfdecaf admin admin] PATH_INFO: `/v3/users/4012c27d84474dc1930ace9990b2bd8b` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:43.533014 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7cd789-e541-4c00-8f4a-7f4dbcfdecaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=4012c27d84474dc1930ace9990b2bd8b)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:43.533928 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f9e4e8-c500-4c9e-a8bc-c4f4801e47cf admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:43.534943 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea7cd789-e541-4c00-8f4a-7f4dbcfdecaf admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:43.625650 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 896/1829] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:13:43 2022] POST /identity/v3/users => generated 299 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:43.630881 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f9e4e8-c500-4c9e-a8bc-c4f4801e47cf admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:43.666854 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f9e4e8-c500-4c9e-a8bc-c4f4801e47cf admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:43.693925 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f3a0c-e1ec-49f3-a247-5ac35d8c7e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:43.693925 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f3a0c-e1ec-49f3-a247-5ac35d8c7e83 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:43.693925 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f3a0c-e1ec-49f3-a247-5ac35d8c7e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:43.693925 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916f3a0c-e1ec-49f3-a247-5ac35d8c7e83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:43.694516 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916f3a0c-e1ec-49f3-a247-5ac35d8c7e83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:43.696682 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916f3a0c-e1ec-49f3-a247-5ac35d8c7e83 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:43.738936 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 897/1830] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:43 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:43.744626 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f3a0c-e1ec-49f3-a247-5ac35d8c7e83 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:43.766163 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916f3a0c-e1ec-49f3-a247-5ac35d8c7e83 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:43.780011 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eea27c-f1f2-4540-bd38-36b1834c1103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:43.782224 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eea27c-f1f2-4540-bd38-36b1834c1103 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:43.785265 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eea27c-f1f2-4540-bd38-36b1834c1103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:43.785586 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15eea27c-f1f2-4540-bd38-36b1834c1103 admin admin] PATH_INFO: `/v3/projects/ea72b8902da644f39a2fa31441a7152c/users/eaf3a7d6b53c4b2fa207d2051d1fc943/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:43.835547 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-ea7cd789-e541-4c00-8f4a-7f4dbcfdecaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4012c27d84474dc1930ace9990b2bd8b'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:43.846417 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-ea7cd789-e541-4c00-8f4a-7f4dbcfdecaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4012c27d84474dc1930ace9990b2bd8b'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:43.847238 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-ea7cd789-e541-4c00-8f4a-7f4dbcfdecaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4012c27d84474dc1930ace9990b2bd8b'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:43.849488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eea27c-f1f2-4540-bd38-36b1834c1103 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea72b8902da644f39a2fa31441a7152c, user_id=eaf3a7d6b53c4b2fa207d2051d1fc943, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:43.859559 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15eea27c-f1f2-4540-bd38-36b1834c1103 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:43.873178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-ea7cd789-e541-4c00-8f4a-7f4dbcfdecaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4012c27d84474dc1930ace9990b2bd8b'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:44.003317 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-ea7cd789-e541-4c00-8f4a-7f4dbcfdecaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4012c27d84474dc1930ace9990b2bd8b'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:44.010137 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 898/1831] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:43 2022] PUT /identity/v3/projects/ea72b8902da644f39a2fa31441a7152c/users/eaf3a7d6b53c4b2fa207d2051d1fc943/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:44.013591 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 934/1832] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:13:43 2022] DELETE /identity/v3/users/4012c27d84474dc1930ace9990b2bd8b => generated 0 bytes in 677 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:44.014786 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eea27c-f1f2-4540-bd38-36b1834c1103 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:44.017382 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570899b-4038-4dd6-91aa-8ed67480f26c None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:44.017714 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570899b-4038-4dd6-91aa-8ed67480f26c None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:44.018082 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570899b-4038-4dd6-91aa-8ed67480f26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:44.019181 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-15eea27c-f1f2-4540-bd38-36b1834c1103 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:44.070289 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-6570899b-4038-4dd6-91aa-8ed67480f26c None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:44.071570 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-6570899b-4038-4dd6-91aa-8ed67480f26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:44.135370 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15eea27c-f1f2-4540-bd38-36b1834c1103 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:44.211054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b766b6-4d9a-4abb-9685-dd795db05853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:44.211795 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b766b6-4d9a-4abb-9685-dd795db05853 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:44.212326 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b766b6-4d9a-4abb-9685-dd795db05853 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:44.213096 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b766b6-4d9a-4abb-9685-dd795db05853 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:44.213999 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b766b6-4d9a-4abb-9685-dd795db05853 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:44.215803 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b766b6-4d9a-4abb-9685-dd795db05853 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:44.234485 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 899/1833] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:44.234485 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b766b6-4d9a-4abb-9685-dd795db05853 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:44.235470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 935/1834] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:44.238024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-19b766b6-4d9a-4abb-9685-dd795db05853 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:44.239517 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570899b-4038-4dd6-91aa-8ed67480f26c None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:44.266005 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570899b-4038-4dd6-91aa-8ed67480f26c None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:44.277756 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c5a5a-774e-4710-83f2-ecee4e2cdeee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:44.278756 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c5a5a-774e-4710-83f2-ecee4e2cdeee admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:44.279027 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c5a5a-774e-4710-83f2-ecee4e2cdeee admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:44.279355 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c5a5a-774e-4710-83f2-ecee4e2cdeee admin admin] PATH_INFO: `/v3/projects/ea72b8902da644f39a2fa31441a7152c/users/eaf3a7d6b53c4b2fa207d2051d1fc943/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:44.282943 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2c5a5a-774e-4710-83f2-ecee4e2cdeee admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea72b8902da644f39a2fa31441a7152c, user_id=eaf3a7d6b53c4b2fa207d2051d1fc943, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:44.291434 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2c5a5a-774e-4710-83f2-ecee4e2cdeee admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:44.320550 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 936/1835] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:44 2022] PUT /identity/v3/projects/ea72b8902da644f39a2fa31441a7152c/users/eaf3a7d6b53c4b2fa207d2051d1fc943/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:44.332960 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573ee72-fb62-4e84-9b9e-89d1b658d4d2 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:44.333740 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573ee72-fb62-4e84-9b9e-89d1b658d4d2 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:44.334695 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573ee72-fb62-4e84-9b9e-89d1b658d4d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:44.355549 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b766b6-4d9a-4abb-9685-dd795db05853 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:44.392835 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-0573ee72-fb62-4e84-9b9e-89d1b658d4d2 None None] MFA Rules not processed for user `eaf3a7d6b53c4b2fa207d2051d1fc943`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:44.396558 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-0573ee72-fb62-4e84-9b9e-89d1b658d4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:44.472091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50a150-1c22-46ec-8f0e-a0155d0bcafb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:44.473278 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50a150-1c22-46ec-8f0e-a0155d0bcafb service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:44.473721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50a150-1c22-46ec-8f0e-a0155d0bcafb service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:44.474215 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e50a150-1c22-46ec-8f0e-a0155d0bcafb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:44.494646 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4e50a150-1c22-46ec-8f0e-a0155d0bcafb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:44.541100 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 937/1836] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:44 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:44.543172 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0573ee72-fb62-4e84-9b9e-89d1b658d4d2 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:44.545991 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-0573ee72-fb62-4e84-9b9e-89d1b658d4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:44.655824 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e50a150-1c22-46ec-8f0e-a0155d0bcafb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:44.657932 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e50a150-1c22-46ec-8f0e-a0155d0bcafb service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:44.671651 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 900/1837] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:44 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:13:44.672388 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50a150-1c22-46ec-8f0e-a0155d0bcafb service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:44.673889 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4e50a150-1c22-46ec-8f0e-a0155d0bcafb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:44.685429 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e50a150-1c22-46ec-8f0e-a0155d0bcafb service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:44.690626 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680191a6-7690-4664-8467-8915084d8238 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:44.691442 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680191a6-7690-4664-8467-8915084d8238 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:44.691562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0573ee72-fb62-4e84-9b9e-89d1b658d4d2 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:44.691699 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680191a6-7690-4664-8467-8915084d8238 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:44.692559 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680191a6-7690-4664-8467-8915084d8238 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:44.697268 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:44.701262 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f6c28f-5c04-4997-b999-233afc6b85b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:44.712278 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f6c28f-5c04-4997-b999-233afc6b85b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:44.712278 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f6c28f-5c04-4997-b999-233afc6b85b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:44.712278 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f6c28f-5c04-4997-b999-233afc6b85b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:44.712618 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680191a6-7690-4664-8467-8915084d8238 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:44.712618 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680191a6-7690-4664-8467-8915084d8238 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:44.727319 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 901/1838] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:44 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:13:44.785689 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f6c28f-5c04-4997-b999-233afc6b85b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:44.787137 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f6c28f-5c04-4997-b999-233afc6b85b9 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:44.809697 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 938/1839] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:13:44.956061 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680191a6-7690-4664-8467-8915084d8238 service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:44.957573 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-680191a6-7690-4664-8467-8915084d8238 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:44.989933 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680191a6-7690-4664-8467-8915084d8238 service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:45.001552 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202d4660-8c33-48c8-8333-edce38ffc255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:45.002593 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d4660-8c33-48c8-8333-edce38ffc255 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:45.002962 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d4660-8c33-48c8-8333-edce38ffc255 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:45.003395 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202d4660-8c33-48c8-8333-edce38ffc255 admin admin] PATH_INFO: `/v3/users/b3f21c541f164c58b7e6688edf32b577` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:45.005078 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202d4660-8c33-48c8-8333-edce38ffc255 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3f21c541f164c58b7e6688edf32b577)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:45.007229 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202d4660-8c33-48c8-8333-edce38ffc255 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:45.325574 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-202d4660-8c33-48c8-8333-edce38ffc255 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3f21c541f164c58b7e6688edf32b577'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:45.365962 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-202d4660-8c33-48c8-8333-edce38ffc255 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3f21c541f164c58b7e6688edf32b577'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:45.383895 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-202d4660-8c33-48c8-8333-edce38ffc255 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3f21c541f164c58b7e6688edf32b577'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:45.386569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-202d4660-8c33-48c8-8333-edce38ffc255 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3f21c541f164c58b7e6688edf32b577'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:45.614613 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-202d4660-8c33-48c8-8333-edce38ffc255 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3f21c541f164c58b7e6688edf32b577'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:45.628500 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f6c28f-5c04-4997-b999-233afc6b85b9 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:45.630573 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-30f6c28f-5c04-4997-b999-233afc6b85b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:45.632547 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 902/1840] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:13:44 2022] DELETE /identity/v3/users/b3f21c541f164c58b7e6688edf32b577 => generated 0 bytes in 677 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:45.637241 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb9f7de-813f-4590-acba-3bba7e94596d None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:45.638302 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb9f7de-813f-4590-acba-3bba7e94596d None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:45.638730 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb9f7de-813f-4590-acba-3bba7e94596d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:45.667767 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f6c28f-5c04-4997-b999-233afc6b85b9 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:45.700340 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-9bb9f7de-813f-4590-acba-3bba7e94596d None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:45.703839 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-9bb9f7de-813f-4590-acba-3bba7e94596d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:45.776022 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f843fd-3ad5-4799-9aa7-f98ed56ba672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:45.777189 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f843fd-3ad5-4799-9aa7-f98ed56ba672 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:45.777417 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f843fd-3ad5-4799-9aa7-f98ed56ba672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:45.777711 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f843fd-3ad5-4799-9aa7-f98ed56ba672 admin admin] PATH_INFO: `/v3/projects/0f928ce25a9d4c419769458770d9c7cc` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:45.778930 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f843fd-3ad5-4799-9aa7-f98ed56ba672 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f928ce25a9d4c419769458770d9c7cc)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:45.781753 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f843fd-3ad5-4799-9aa7-f98ed56ba672 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:45.785902 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:45.792149 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 903/1841] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:45.805258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb9f7de-813f-4590-acba-3bba7e94596d None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:45.806819 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-9bb9f7de-813f-4590-acba-3bba7e94596d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:45.869154 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb9f7de-813f-4590-acba-3bba7e94596d None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:45.872848 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ede91-9eef-4968-bd11-d8ec4397994d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:45.873506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ede91-9eef-4968-bd11-d8ec4397994d service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:45.873897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ede91-9eef-4968-bd11-d8ec4397994d service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:45.874175 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ede91-9eef-4968-bd11-d8ec4397994d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:45.884197 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58ede91-9eef-4968-bd11-d8ec4397994d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:45.885983 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58ede91-9eef-4968-bd11-d8ec4397994d service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:45.897385 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 904/1842] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:13:45.904109 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-67f843fd-3ad5-4799-9aa7-f98ed56ba672 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f928ce25a9d4c419769458770d9c7cc', 'request_id': 'req-67f843fd-3ad5-4799-9aa7-f98ed56ba672'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:45.924135 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-67f843fd-3ad5-4799-9aa7-f98ed56ba672 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f928ce25a9d4c419769458770d9c7cc', 'request_id': 'req-67f843fd-3ad5-4799-9aa7-f98ed56ba672'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:45.940365 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-67f843fd-3ad5-4799-9aa7-f98ed56ba672 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f928ce25a9d4c419769458770d9c7cc', 'request_id': 'req-67f843fd-3ad5-4799-9aa7-f98ed56ba672'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:46.077669 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-67f843fd-3ad5-4799-9aa7-f98ed56ba672 admin admin] The token cache is being invalidate because project 0f928ce25a9d4c419769458770d9c7cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:13:46.078071 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-67f843fd-3ad5-4799-9aa7-f98ed56ba672 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:46.078911 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-67f843fd-3ad5-4799-9aa7-f98ed56ba672 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:46.081165 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 939/1843] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:13:45 2022] DELETE /identity/v3/projects/0f928ce25a9d4c419769458770d9c7cc => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:46.085261 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ede91-9eef-4968-bd11-d8ec4397994d service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:46.086679 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-a58ede91-9eef-4968-bd11-d8ec4397994d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:46.099685 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ede91-9eef-4968-bd11-d8ec4397994d service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:46.158707 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a102331-5b2c-4247-a744-825a445b86b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:46.159627 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a102331-5b2c-4247-a744-825a445b86b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:46.159946 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a102331-5b2c-4247-a744-825a445b86b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:46.160394 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a102331-5b2c-4247-a744-825a445b86b8 admin admin] PATH_INFO: `/v3/users/e8d2015ff7144a08aecfd1a2668d3809` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:46.161632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a102331-5b2c-4247-a744-825a445b86b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8d2015ff7144a08aecfd1a2668d3809)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:46.163868 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a102331-5b2c-4247-a744-825a445b86b8 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:46.516912 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f843fd-3ad5-4799-9aa7-f98ed56ba672 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:46.518929 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-67f843fd-3ad5-4799-9aa7-f98ed56ba672 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:46.536403 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-4a102331-5b2c-4247-a744-825a445b86b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8d2015ff7144a08aecfd1a2668d3809'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:46.554580 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-4a102331-5b2c-4247-a744-825a445b86b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8d2015ff7144a08aecfd1a2668d3809'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:46.563284 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f843fd-3ad5-4799-9aa7-f98ed56ba672 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:46.567212 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-4a102331-5b2c-4247-a744-825a445b86b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8d2015ff7144a08aecfd1a2668d3809'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:46.568398 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-4a102331-5b2c-4247-a744-825a445b86b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8d2015ff7144a08aecfd1a2668d3809'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:46.570262 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2b64b3-d655-41f1-add6-6d8aaa0d7703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:46.571334 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b64b3-d655-41f1-add6-6d8aaa0d7703 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:46.571638 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b64b3-d655-41f1-add6-6d8aaa0d7703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:46.572161 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2b64b3-d655-41f1-add6-6d8aaa0d7703 admin admin] PATH_INFO: `/v3/projects/d1565e79ae634de0adc621de33ef96ab` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:46.575803 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2b64b3-d655-41f1-add6-6d8aaa0d7703 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1565e79ae634de0adc621de33ef96ab)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:46.576611 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2b64b3-d655-41f1-add6-6d8aaa0d7703 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:46.631771 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-4a102331-5b2c-4247-a744-825a445b86b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8d2015ff7144a08aecfd1a2668d3809'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:46.639616 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 905/1844] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:13:46 2022] DELETE /identity/v3/users/e8d2015ff7144a08aecfd1a2668d3809 => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:46.740874 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2f2b64b3-d655-41f1-add6-6d8aaa0d7703 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1565e79ae634de0adc621de33ef96ab', 'request_id': 'req-2f2b64b3-d655-41f1-add6-6d8aaa0d7703'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:46.749617 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2f2b64b3-d655-41f1-add6-6d8aaa0d7703 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1565e79ae634de0adc621de33ef96ab', 'request_id': 'req-2f2b64b3-d655-41f1-add6-6d8aaa0d7703'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:46.757360 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2f2b64b3-d655-41f1-add6-6d8aaa0d7703 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1565e79ae634de0adc621de33ef96ab', 'request_id': 'req-2f2b64b3-d655-41f1-add6-6d8aaa0d7703'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:46.832404 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2f2b64b3-d655-41f1-add6-6d8aaa0d7703 admin admin] The token cache is being invalidate because project d1565e79ae634de0adc621de33ef96ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:13:46.832739 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2f2b64b3-d655-41f1-add6-6d8aaa0d7703 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:46.840339 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2f2b64b3-d655-41f1-add6-6d8aaa0d7703 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:46.840339 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 940/1845] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:13:46 2022] DELETE /identity/v3/projects/d1565e79ae634de0adc621de33ef96ab => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:46.841476 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3477d097-86bf-44d7-af51-2bea20eb53d0 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:46.841772 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3477d097-86bf-44d7-af51-2bea20eb53d0 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:46.842077 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3477d097-86bf-44d7-af51-2bea20eb53d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:46.862384 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-3477d097-86bf-44d7-af51-2bea20eb53d0 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:46.863614 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-3477d097-86bf-44d7-af51-2bea20eb53d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:46.935765 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 906/1846] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:13:46.943911 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe460f74-48e9-437d-bbe2-1cf44518b2ea None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:46.944209 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe460f74-48e9-437d-bbe2-1cf44518b2ea None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:46.944493 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe460f74-48e9-437d-bbe2-1cf44518b2ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:46.963568 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-fe460f74-48e9-437d-bbe2-1cf44518b2ea None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:46.964832 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-fe460f74-48e9-437d-bbe2-1cf44518b2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:46.974413 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 941/1847] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:46.978801 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3477d097-86bf-44d7-af51-2bea20eb53d0 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:46.992329 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3477d097-86bf-44d7-af51-2bea20eb53d0 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:46.997380 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d84e1f-6156-4d45-88f8-19d2c6979b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:46.998088 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d84e1f-6156-4d45-88f8-19d2c6979b63 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:46.998373 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d84e1f-6156-4d45-88f8-19d2c6979b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:47.006290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d84e1f-6156-4d45-88f8-19d2c6979b63 admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:47.007475 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d84e1f-6156-4d45-88f8-19d2c6979b63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:13:47.007755 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d84e1f-6156-4d45-88f8-19d2c6979b63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:47.009472 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d84e1f-6156-4d45-88f8-19d2c6979b63 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:47.023905 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 907/1848] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:13:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:13:47.027028 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe460f74-48e9-437d-bbe2-1cf44518b2ea None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:47.037600 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe460f74-48e9-437d-bbe2-1cf44518b2ea None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:47.043685 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed417bc8-befa-482a-989b-9b6853343d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:47.046193 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed417bc8-befa-482a-989b-9b6853343d39 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:47.046193 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed417bc8-befa-482a-989b-9b6853343d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:47.046193 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed417bc8-befa-482a-989b-9b6853343d39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:47.046193 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed417bc8-befa-482a-989b-9b6853343d39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:47.048301 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed417bc8-befa-482a-989b-9b6853343d39 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:47.202941 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 942/1849] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:13:47 2022] POST /identity/v3/projects => generated 377 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:47.203873 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d84e1f-6156-4d45-88f8-19d2c6979b63 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:47.214439 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d84e1f-6156-4d45-88f8-19d2c6979b63 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:47.304599 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47bd707-5c09-4067-8d55-5c25e39e9dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:47.310424 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47bd707-5c09-4067-8d55-5c25e39e9dd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:47.310424 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47bd707-5c09-4067-8d55-5c25e39e9dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:47.310424 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47bd707-5c09-4067-8d55-5c25e39e9dd6 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:47.310424 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47bd707-5c09-4067-8d55-5c25e39e9dd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:47.310424 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47bd707-5c09-4067-8d55-5c25e39e9dd6 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:47.419468 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 908/1850] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:13:47 2022] POST /identity/v3/users => generated 289 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:47.419905 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed417bc8-befa-482a-989b-9b6853343d39 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:47.419905 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-ed417bc8-befa-482a-989b-9b6853343d39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:47.420628 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47bd707-5c09-4067-8d55-5c25e39e9dd6 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:47.430754 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47bd707-5c09-4067-8d55-5c25e39e9dd6 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:47.437942 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2adef28-7cfc-4e4f-a22c-d9a7f880c0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:47.438451 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2adef28-7cfc-4e4f-a22c-d9a7f880c0f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:47.438892 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2adef28-7cfc-4e4f-a22c-d9a7f880c0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:47.439326 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2adef28-7cfc-4e4f-a22c-d9a7f880c0f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:47.440606 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2adef28-7cfc-4e4f-a22c-d9a7f880c0f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:47.442428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2adef28-7cfc-4e4f-a22c-d9a7f880c0f8 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:47.443544 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed417bc8-befa-482a-989b-9b6853343d39 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:47.455835 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 909/1851] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:13:47.457648 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2adef28-7cfc-4e4f-a22c-d9a7f880c0f8 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:47.459682 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f5e72-a561-4368-b3f5-ede6feb7eac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:47.460181 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-f2adef28-7cfc-4e4f-a22c-d9a7f880c0f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:47.460749 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f5e72-a561-4368-b3f5-ede6feb7eac1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:47.460955 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f5e72-a561-4368-b3f5-ede6feb7eac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:47.461233 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f5e72-a561-4368-b3f5-ede6feb7eac1 admin admin] PATH_INFO: `/v3/projects/a8ace4333a914c72a33cbdfd5c8d6835` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:47.463224 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f5e72-a561-4368-b3f5-ede6feb7eac1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8ace4333a914c72a33cbdfd5c8d6835)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:47.465200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f5e72-a561-4368-b3f5-ede6feb7eac1 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:47.475974 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2adef28-7cfc-4e4f-a22c-d9a7f880c0f8 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:47.482165 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e8629f-2182-4f0c-973e-e4be88de057c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:47.483427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e8629f-2182-4f0c-973e-e4be88de057c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:47.483893 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e8629f-2182-4f0c-973e-e4be88de057c admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:47.484608 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e8629f-2182-4f0c-973e-e4be88de057c admin admin] PATH_INFO: `/v3/users/eb472b67fcf54a4ebe1be96a7d1bd4d4` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:47.485990 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e8629f-2182-4f0c-973e-e4be88de057c admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb472b67fcf54a4ebe1be96a7d1bd4d4)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:47.488111 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e8629f-2182-4f0c-973e-e4be88de057c admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:47.582498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1b1f5e72-a561-4368-b3f5-ede6feb7eac1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8ace4333a914c72a33cbdfd5c8d6835', 'request_id': 'req-1b1f5e72-a561-4368-b3f5-ede6feb7eac1'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:47.594563 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1b1f5e72-a561-4368-b3f5-ede6feb7eac1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8ace4333a914c72a33cbdfd5c8d6835', 'request_id': 'req-1b1f5e72-a561-4368-b3f5-ede6feb7eac1'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:47.598343 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1b1f5e72-a561-4368-b3f5-ede6feb7eac1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8ace4333a914c72a33cbdfd5c8d6835', 'request_id': 'req-1b1f5e72-a561-4368-b3f5-ede6feb7eac1'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:47.677513 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1b1f5e72-a561-4368-b3f5-ede6feb7eac1 admin admin] The token cache is being invalidate because project a8ace4333a914c72a33cbdfd5c8d6835 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:13:47.677718 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1b1f5e72-a561-4368-b3f5-ede6feb7eac1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:47.678634 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1b1f5e72-a561-4368-b3f5-ede6feb7eac1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:47.680335 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 943/1852] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:13:47 2022] DELETE /identity/v3/projects/a8ace4333a914c72a33cbdfd5c8d6835 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:13:47.681261 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f5e72-a561-4368-b3f5-ede6feb7eac1 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:47.683063 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-1b1f5e72-a561-4368-b3f5-ede6feb7eac1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:47.689204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-50e8629f-2182-4f0c-973e-e4be88de057c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb472b67fcf54a4ebe1be96a7d1bd4d4'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:47.702580 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-50e8629f-2182-4f0c-973e-e4be88de057c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb472b67fcf54a4ebe1be96a7d1bd4d4'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:47.711894 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-50e8629f-2182-4f0c-973e-e4be88de057c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb472b67fcf54a4ebe1be96a7d1bd4d4'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:47.712685 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-50e8629f-2182-4f0c-973e-e4be88de057c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb472b67fcf54a4ebe1be96a7d1bd4d4'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:47.760438 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f5e72-a561-4368-b3f5-ede6feb7eac1 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:47.765808 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3899d4f4-505b-4ca4-a5c2-5a183a4ba069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:47.766491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3899d4f4-505b-4ca4-a5c2-5a183a4ba069 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:47.766778 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3899d4f4-505b-4ca4-a5c2-5a183a4ba069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:47.767060 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3899d4f4-505b-4ca4-a5c2-5a183a4ba069 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:47.767829 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3899d4f4-505b-4ca4-a5c2-5a183a4ba069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:47.769432 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3899d4f4-505b-4ca4-a5c2-5a183a4ba069 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:47.781398 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 944/1853] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:47.787994 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149e1d9-b6aa-436c-b6d9-8f5941ef188b None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:47.787994 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149e1d9-b6aa-436c-b6d9-8f5941ef188b None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:47.787994 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8149e1d9-b6aa-436c-b6d9-8f5941ef188b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:47.808552 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-50e8629f-2182-4f0c-973e-e4be88de057c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb472b67fcf54a4ebe1be96a7d1bd4d4'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:47.812208 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-8149e1d9-b6aa-436c-b6d9-8f5941ef188b None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:47.813586 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-8149e1d9-b6aa-436c-b6d9-8f5941ef188b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:47.823087 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 910/1854] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:13:47 2022] DELETE /identity/v3/users/eb472b67fcf54a4ebe1be96a7d1bd4d4 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:47.824147 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 945/1855] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:47.824925 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e8629f-2182-4f0c-973e-e4be88de057c admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:47.825871 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca388b3-2e85-495e-800a-ee8a06b01fbd None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:47.826388 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca388b3-2e85-495e-800a-ee8a06b01fbd None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:47.827337 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca388b3-2e85-495e-800a-ee8a06b01fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:47.827938 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-50e8629f-2182-4f0c-973e-e4be88de057c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:47.846230 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-1ca388b3-2e85-495e-800a-ee8a06b01fbd None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:47.847447 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-1ca388b3-2e85-495e-800a-ee8a06b01fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:47.899096 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e8629f-2182-4f0c-973e-e4be88de057c admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:47.912171 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 946/1856] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:47.915461 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92717e4b-21a1-4d2d-a0d1-593fc0874054 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:47.915461 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92717e4b-21a1-4d2d-a0d1-593fc0874054 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:47.915657 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f4bc2e-73d0-47ac-8348-1f7fd3576be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:47.915797 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92717e4b-21a1-4d2d-a0d1-593fc0874054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:47.915983 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4bc2e-73d0-47ac-8348-1f7fd3576be1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:47.915983 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4bc2e-73d0-47ac-8348-1f7fd3576be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:47.915983 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f4bc2e-73d0-47ac-8348-1f7fd3576be1 admin admin] PATH_INFO: `/v3/projects/90cf8824908344a687ee5828e257da3c/users/b0d3a6765ea9469ebd3e526f54904803/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:47.933298 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f4bc2e-73d0-47ac-8348-1f7fd3576be1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90cf8824908344a687ee5828e257da3c, user_id=b0d3a6765ea9469ebd3e526f54904803, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:47.935483 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-92717e4b-21a1-4d2d-a0d1-593fc0874054 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:47.936754 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f4bc2e-73d0-47ac-8348-1f7fd3576be1 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:47.937440 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-92717e4b-21a1-4d2d-a0d1-593fc0874054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:47.946624 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 947/1857] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:47.947543 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92717e4b-21a1-4d2d-a0d1-593fc0874054 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:47.949697 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-92717e4b-21a1-4d2d-a0d1-593fc0874054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:48.013618 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92717e4b-21a1-4d2d-a0d1-593fc0874054 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:48.016979 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 911/1858] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:47 2022] PUT /identity/v3/projects/90cf8824908344a687ee5828e257da3c/users/b0d3a6765ea9469ebd3e526f54904803/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:48.017940 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f4bc2e-73d0-47ac-8348-1f7fd3576be1 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:48.085481 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed2bba8-3808-498c-8b00-69cde1d8f52b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:48.086431 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed2bba8-3808-498c-8b00-69cde1d8f52b service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:48.086890 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed2bba8-3808-498c-8b00-69cde1d8f52b service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:48.087090 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed2bba8-3808-498c-8b00-69cde1d8f52b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:48.089046 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f4bc2e-73d0-47ac-8348-1f7fd3576be1 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:48.091836 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:48.096415 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d78ad9-c696-41d8-a21e-0194c4d69061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:48.098822 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d78ad9-c696-41d8-a21e-0194c4d69061 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:48.098822 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d78ad9-c696-41d8-a21e-0194c4d69061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:48.098822 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d78ad9-c696-41d8-a21e-0194c4d69061 admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:48.098822 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d78ad9-c696-41d8-a21e-0194c4d69061 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:13:48.099304 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d78ad9-c696-41d8-a21e-0194c4d69061 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:48.100404 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed2bba8-3808-498c-8b00-69cde1d8f52b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:48.101023 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d78ad9-c696-41d8-a21e-0194c4d69061 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:48.102656 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed2bba8-3808-498c-8b00-69cde1d8f52b service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:48.115042 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 912/1859] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:13:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:13:48.118762 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3ccec-35b2-4637-ab26-b57a58b30ee2 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:48.119176 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3ccec-35b2-4637-ab26-b57a58b30ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:48.120197 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d3ccec-35b2-4637-ab26-b57a58b30ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:48.121560 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 948/1860] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:13:48.122535 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed2bba8-3808-498c-8b00-69cde1d8f52b service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:48.136116 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed2bba8-3808-498c-8b00-69cde1d8f52b service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:48.143276 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40501d84-fdc5-4d4a-9e72-a9a085910c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:48.144523 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40501d84-fdc5-4d4a-9e72-a9a085910c25 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:48.144737 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40501d84-fdc5-4d4a-9e72-a9a085910c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:48.145184 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40501d84-fdc5-4d4a-9e72-a9a085910c25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:48.146296 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40501d84-fdc5-4d4a-9e72-a9a085910c25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:48.148397 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40501d84-fdc5-4d4a-9e72-a9a085910c25 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:48.178928 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-66d3ccec-35b2-4637-ab26-b57a58b30ee2 None None] MFA Rules not processed for user `b0d3a6765ea9469ebd3e526f54904803`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:48.180636 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-66d3ccec-35b2-4637-ab26-b57a58b30ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:48.260817 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 913/1861] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:48.266456 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3ccec-35b2-4637-ab26-b57a58b30ee2 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:48.275184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d3ccec-35b2-4637-ab26-b57a58b30ee2 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:48.280284 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbd6887-c6e3-4520-8f1d-2c3262dd9416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:48.280924 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd6887-c6e3-4520-8f1d-2c3262dd9416 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:48.281198 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd6887-c6e3-4520-8f1d-2c3262dd9416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:48.281476 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbd6887-c6e3-4520-8f1d-2c3262dd9416 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:48.282157 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbd6887-c6e3-4520-8f1d-2c3262dd9416 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:48.284837 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbd6887-c6e3-4520-8f1d-2c3262dd9416 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:48.301497 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 949/1862] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:13:48 2022] POST /identity/v3/projects => generated 375 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:48.304148 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40501d84-fdc5-4d4a-9e72-a9a085910c25 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:48.413410 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40501d84-fdc5-4d4a-9e72-a9a085910c25 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:48.424916 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd85521-43f4-4030-b928-3591e3860acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:48.430876 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd85521-43f4-4030-b928-3591e3860acb admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:48.430876 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd85521-43f4-4030-b928-3591e3860acb admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:48.430876 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd85521-43f4-4030-b928-3591e3860acb admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:48.430876 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd85521-43f4-4030-b928-3591e3860acb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:48.430876 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd85521-43f4-4030-b928-3591e3860acb admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:48.452574 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:48.461586 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 914/1863] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:13:48 2022] POST /identity/v3/projects => generated 377 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:13:48.465726 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbd6887-c6e3-4520-8f1d-2c3262dd9416 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:48.537635 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbd6887-c6e3-4520-8f1d-2c3262dd9416 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:48.542654 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18692663-b5b3-4783-9a71-ce40b4387f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:48.543331 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18692663-b5b3-4783-9a71-ce40b4387f5b admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:48.543620 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18692663-b5b3-4783-9a71-ce40b4387f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:48.543935 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18692663-b5b3-4783-9a71-ce40b4387f5b admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:48.544624 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18692663-b5b3-4783-9a71-ce40b4387f5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:48.550011 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 950/1864] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:13:48 2022] POST /identity/v3/users => generated 288 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:13:48.550151 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18692663-b5b3-4783-9a71-ce40b4387f5b admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:48.551010 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd85521-43f4-4030-b928-3591e3860acb admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:48.561173 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd85521-43f4-4030-b928-3591e3860acb admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:48.565718 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af88ce68-9378-45ad-ae4a-68c9caf396cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:48.566543 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88ce68-9378-45ad-ae4a-68c9caf396cc admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:48.578741 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88ce68-9378-45ad-ae4a-68c9caf396cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:48.578741 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af88ce68-9378-45ad-ae4a-68c9caf396cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:48.578741 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af88ce68-9378-45ad-ae4a-68c9caf396cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:48.578741 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af88ce68-9378-45ad-ae4a-68c9caf396cc admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:48.588759 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 951/1865] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:13:48.589764 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af88ce68-9378-45ad-ae4a-68c9caf396cc admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:48.591676 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-af88ce68-9378-45ad-ae4a-68c9caf396cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:48.609518 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af88ce68-9378-45ad-ae4a-68c9caf396cc admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:48.613381 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b19ccfb-88c6-4f61-910a-dfe20897b6dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:48.614265 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b19ccfb-88c6-4f61-910a-dfe20897b6dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:48.614544 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b19ccfb-88c6-4f61-910a-dfe20897b6dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:48.614816 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b19ccfb-88c6-4f61-910a-dfe20897b6dd admin admin] PATH_INFO: `/v3/projects/085e5740a6c84bb6918d9a3fe93a8852` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:48.616009 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b19ccfb-88c6-4f61-910a-dfe20897b6dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=085e5740a6c84bb6918d9a3fe93a8852)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:48.617900 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b19ccfb-88c6-4f61-910a-dfe20897b6dd admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:48.660973 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 915/1866] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:13:48 2022] POST /identity/v3/users => generated 289 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:13:48.662551 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18692663-b5b3-4783-9a71-ce40b4387f5b admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:48.673142 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18692663-b5b3-4783-9a71-ce40b4387f5b admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:48.677842 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4388e57-d71e-48a0-b44a-ed06b6a47c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:48.678506 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4388e57-d71e-48a0-b44a-ed06b6a47c03 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:48.678790 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4388e57-d71e-48a0-b44a-ed06b6a47c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:48.679063 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4388e57-d71e-48a0-b44a-ed06b6a47c03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:48.679820 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4388e57-d71e-48a0-b44a-ed06b6a47c03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:48.681450 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4388e57-d71e-48a0-b44a-ed06b6a47c03 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:48.697117 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 916/1867] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:48.697789 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4388e57-d71e-48a0-b44a-ed06b6a47c03 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:48.709213 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4388e57-d71e-48a0-b44a-ed06b6a47c03 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:48.716909 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e164d3c9-c06f-4a7d-9419-029ddb3273c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:48.718440 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e164d3c9-c06f-4a7d-9419-029ddb3273c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:48.718995 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e164d3c9-c06f-4a7d-9419-029ddb3273c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:48.720130 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e164d3c9-c06f-4a7d-9419-029ddb3273c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:48.720904 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e164d3c9-c06f-4a7d-9419-029ddb3273c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:48.722527 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e164d3c9-c06f-4a7d-9419-029ddb3273c5 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:48.746201 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 917/1868] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:48.747408 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e164d3c9-c06f-4a7d-9419-029ddb3273c5 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:48.750010 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-6b19ccfb-88c6-4f61-910a-dfe20897b6dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '085e5740a6c84bb6918d9a3fe93a8852', 'request_id': 'req-6b19ccfb-88c6-4f61-910a-dfe20897b6dd'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:48.758619 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-6b19ccfb-88c6-4f61-910a-dfe20897b6dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '085e5740a6c84bb6918d9a3fe93a8852', 'request_id': 'req-6b19ccfb-88c6-4f61-910a-dfe20897b6dd'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:48.767421 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-6b19ccfb-88c6-4f61-910a-dfe20897b6dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '085e5740a6c84bb6918d9a3fe93a8852', 'request_id': 'req-6b19ccfb-88c6-4f61-910a-dfe20897b6dd'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:48.771765 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e164d3c9-c06f-4a7d-9419-029ddb3273c5 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:48.841190 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e4b686-79ba-44e8-b75d-d0edc75109bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:48.842099 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4b686-79ba-44e8-b75d-d0edc75109bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:48.842470 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4b686-79ba-44e8-b75d-d0edc75109bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:48.842754 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e4b686-79ba-44e8-b75d-d0edc75109bd admin admin] PATH_INFO: `/v3/projects/0e0b3c17322848d58f6d2984428b99ea/users/780716e0cd604f2aa6ee9bd8b0ca645f/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:48.857759 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e4b686-79ba-44e8-b75d-d0edc75109bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e0b3c17322848d58f6d2984428b99ea, user_id=780716e0cd604f2aa6ee9bd8b0ca645f, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:48.861312 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e4b686-79ba-44e8-b75d-d0edc75109bd admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:48.894591 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-6b19ccfb-88c6-4f61-910a-dfe20897b6dd admin admin] The token cache is being invalidate because project 085e5740a6c84bb6918d9a3fe93a8852 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:13:48.896589 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-6b19ccfb-88c6-4f61-910a-dfe20897b6dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:48.896589 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-6b19ccfb-88c6-4f61-910a-dfe20897b6dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:48.897184 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 952/1869] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:13:48 2022] DELETE /identity/v3/projects/085e5740a6c84bb6918d9a3fe93a8852 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:48.897938 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b19ccfb-88c6-4f61-910a-dfe20897b6dd admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:48.899267 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-6b19ccfb-88c6-4f61-910a-dfe20897b6dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:48.911758 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b19ccfb-88c6-4f61-910a-dfe20897b6dd admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:48.916406 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3561a13-2187-4ade-be2a-771fba3daa43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:48.917047 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3561a13-2187-4ade-be2a-771fba3daa43 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:48.917330 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3561a13-2187-4ade-be2a-771fba3daa43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:48.917605 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3561a13-2187-4ade-be2a-771fba3daa43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:48.918318 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3561a13-2187-4ade-be2a-771fba3daa43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:48.919941 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3561a13-2187-4ade-be2a-771fba3daa43 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:48.930129 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 953/1870] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:48.930895 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3561a13-2187-4ade-be2a-771fba3daa43 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:48.934211 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f3561a13-2187-4ade-be2a-771fba3daa43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:48.934724 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 918/1871] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:48 2022] PUT /identity/v3/projects/0e0b3c17322848d58f6d2984428b99ea/users/780716e0cd604f2aa6ee9bd8b0ca645f/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:48.936908 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e4b686-79ba-44e8-b75d-d0edc75109bd admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:48.947335 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3561a13-2187-4ade-be2a-771fba3daa43 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:49.006730 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf179a4-a952-4376-b661-64e96f2c97cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:49.007613 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf179a4-a952-4376-b661-64e96f2c97cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:49.007926 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf179a4-a952-4376-b661-64e96f2c97cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:49.011342 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf179a4-a952-4376-b661-64e96f2c97cd admin admin] PATH_INFO: `/v3/users/eaf3a7d6b53c4b2fa207d2051d1fc943` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:49.011342 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf179a4-a952-4376-b661-64e96f2c97cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=eaf3a7d6b53c4b2fa207d2051d1fc943)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:49.011342 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf179a4-a952-4376-b661-64e96f2c97cd admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:49.023295 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e4b686-79ba-44e8-b75d-d0edc75109bd admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:49.028517 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41013e93-6ce2-43d3-96dc-25791f3a2bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:49.029613 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41013e93-6ce2-43d3-96dc-25791f3a2bb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:49.030005 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41013e93-6ce2-43d3-96dc-25791f3a2bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:49.030380 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41013e93-6ce2-43d3-96dc-25791f3a2bb3 admin admin] PATH_INFO: `/v3/projects/d669d36a808f47deab3ba1c36267b05e/users/284df019f75544c0ad5ab55233e557ef/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:49.047957 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41013e93-6ce2-43d3-96dc-25791f3a2bb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d669d36a808f47deab3ba1c36267b05e, user_id=284df019f75544c0ad5ab55233e557ef, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:49.050727 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41013e93-6ce2-43d3-96dc-25791f3a2bb3 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:49.142162 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 919/1872] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:48 2022] PUT /identity/v3/projects/d669d36a808f47deab3ba1c36267b05e/users/284df019f75544c0ad5ab55233e557ef/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:49.143787 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50761bb-7f3c-4eab-b798-6d459b517558 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:49.146990 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50761bb-7f3c-4eab-b798-6d459b517558 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:49.146990 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50761bb-7f3c-4eab-b798-6d459b517558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:49.173335 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-c50761bb-7f3c-4eab-b798-6d459b517558 None None] MFA Rules not processed for user `780716e0cd604f2aa6ee9bd8b0ca645f`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:49.174636 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c50761bb-7f3c-4eab-b798-6d459b517558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:49.241734 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 920/1873] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:49.241734 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b4b3b4-f15b-4ffc-bc1f-7f82f1bb2c7b None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:49.241734 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b4b3b4-f15b-4ffc-bc1f-7f82f1bb2c7b None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:49.241734 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b4b3b4-f15b-4ffc-bc1f-7f82f1bb2c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:49.242119 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-fcf179a4-a952-4376-b661-64e96f2c97cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eaf3a7d6b53c4b2fa207d2051d1fc943'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:49.248620 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-fcf179a4-a952-4376-b661-64e96f2c97cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eaf3a7d6b53c4b2fa207d2051d1fc943'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:49.249254 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-fcf179a4-a952-4376-b661-64e96f2c97cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eaf3a7d6b53c4b2fa207d2051d1fc943'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:49.264581 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-fcf179a4-a952-4376-b661-64e96f2c97cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eaf3a7d6b53c4b2fa207d2051d1fc943'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:49.267591 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-e2b4b3b4-f15b-4ffc-bc1f-7f82f1bb2c7b None None] MFA Rules not processed for user `284df019f75544c0ad5ab55233e557ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:49.267854 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e2b4b3b4-f15b-4ffc-bc1f-7f82f1bb2c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:49.339952 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 921/1874] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:49.339952 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b4b3b4-f15b-4ffc-bc1f-7f82f1bb2c7b None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:49.339952 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e2b4b3b4-f15b-4ffc-bc1f-7f82f1bb2c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:49.402259 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-fcf179a4-a952-4376-b661-64e96f2c97cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eaf3a7d6b53c4b2fa207d2051d1fc943'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:49.407942 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b4b3b4-f15b-4ffc-bc1f-7f82f1bb2c7b None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:49.409689 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 954/1875] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:13:48 2022] DELETE /identity/v3/users/eaf3a7d6b53c4b2fa207d2051d1fc943 => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:49.413477 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf179a4-a952-4376-b661-64e96f2c97cd admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:49.414665 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-fcf179a4-a952-4376-b661-64e96f2c97cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:49.469238 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeee6f8-8df4-40f5-ab9b-c370d4dba603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:49.469916 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeee6f8-8df4-40f5-ab9b-c370d4dba603 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:49.470202 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeee6f8-8df4-40f5-ab9b-c370d4dba603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:49.470482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeee6f8-8df4-40f5-ab9b-c370d4dba603 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:49.471207 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeee6f8-8df4-40f5-ab9b-c370d4dba603 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:49.473331 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeee6f8-8df4-40f5-ab9b-c370d4dba603 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:49.485076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf179a4-a952-4376-b661-64e96f2c97cd admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:49.490447 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed7b26-8be4-4eaa-81b6-087dd23e02b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:49.491304 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed7b26-8be4-4eaa-81b6-087dd23e02b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:49.491589 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed7b26-8be4-4eaa-81b6-087dd23e02b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:49.491903 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed7b26-8be4-4eaa-81b6-087dd23e02b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:49.492597 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ed7b26-8be4-4eaa-81b6-087dd23e02b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:49.494502 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ed7b26-8be4-4eaa-81b6-087dd23e02b0 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:49.650414 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 922/1877] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:13:49 2022] POST /identity/v3/projects => generated 377 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:49.652221 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 955/1877] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:13:49 2022] POST /identity/v3/projects => generated 375 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:49.655325 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeee6f8-8df4-40f5-ab9b-c370d4dba603 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:49.655472 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed7b26-8be4-4eaa-81b6-087dd23e02b0 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:49.657115 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-cfeee6f8-8df4-40f5-ab9b-c370d4dba603 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:49.677910 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:49.733615 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed7b26-8be4-4eaa-81b6-087dd23e02b0 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:49.733815 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeee6f8-8df4-40f5-ab9b-c370d4dba603 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:49.738783 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e29bed-3b5b-4662-b1b3-6efc1c6ffb7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:49.739487 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e29bed-3b5b-4662-b1b3-6efc1c6ffb7a admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:49.739796 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e29bed-3b5b-4662-b1b3-6efc1c6ffb7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:49.740076 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e29bed-3b5b-4662-b1b3-6efc1c6ffb7a admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:49.740398 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f190b-a201-4fef-841b-fef76aabf400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:49.740800 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e29bed-3b5b-4662-b1b3-6efc1c6ffb7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:49.741222 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f190b-a201-4fef-841b-fef76aabf400 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:49.741513 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f190b-a201-4fef-841b-fef76aabf400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:49.741820 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7f190b-a201-4fef-841b-fef76aabf400 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:49.742531 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7f190b-a201-4fef-841b-fef76aabf400 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:49.742737 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e29bed-3b5b-4662-b1b3-6efc1c6ffb7a admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:49.744805 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7f190b-a201-4fef-841b-fef76aabf400 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:49.852241 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 923/1878] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:13:49 2022] POST /identity/v3/users => generated 295 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:49.854551 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e29bed-3b5b-4662-b1b3-6efc1c6ffb7a admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:49.865751 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e29bed-3b5b-4662-b1b3-6efc1c6ffb7a admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:49.866813 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 956/1879] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:13:49 2022] POST /identity/v3/users => generated 294 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:49.870494 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f190b-a201-4fef-841b-fef76aabf400 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:49.873864 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb5f2b-200e-4e8a-848d-3cf51dc0edee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:49.874680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb5f2b-200e-4e8a-848d-3cf51dc0edee admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:49.884474 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb5f2b-200e-4e8a-848d-3cf51dc0edee admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:49.885476 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7f190b-a201-4fef-841b-fef76aabf400 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:49.885887 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb5f2b-200e-4e8a-848d-3cf51dc0edee admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:49.886977 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fb5f2b-200e-4e8a-848d-3cf51dc0edee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:49.888741 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fb5f2b-200e-4e8a-848d-3cf51dc0edee admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:49.892020 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b86321-c50d-4111-80fc-813f82de1df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:49.892705 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b86321-c50d-4111-80fc-813f82de1df1 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:49.892991 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b86321-c50d-4111-80fc-813f82de1df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:49.893269 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b86321-c50d-4111-80fc-813f82de1df1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:49.893997 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b86321-c50d-4111-80fc-813f82de1df1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:49.895585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b86321-c50d-4111-80fc-813f82de1df1 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:49.915079 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 957/1880] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:49.915079 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b86321-c50d-4111-80fc-813f82de1df1 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:49.919833 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 924/1881] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:13:49.919833 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb5f2b-200e-4e8a-848d-3cf51dc0edee admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:49.922080 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b0b86321-c50d-4111-80fc-813f82de1df1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:49.924581 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-98fb5f2b-200e-4e8a-848d-3cf51dc0edee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:49.946794 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b86321-c50d-4111-80fc-813f82de1df1 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:49.950306 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb5f2b-200e-4e8a-848d-3cf51dc0edee admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:49.967537 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137eb68a-de7d-4bc8-b06e-e16f03e80b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:49.967640 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ed1304-8e07-4030-8dd4-1ce20656b7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:49.968495 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137eb68a-de7d-4bc8-b06e-e16f03e80b8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:49.968609 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed1304-8e07-4030-8dd4-1ce20656b7d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:49.972351 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed1304-8e07-4030-8dd4-1ce20656b7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:49.972461 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137eb68a-de7d-4bc8-b06e-e16f03e80b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:49.972564 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed1304-8e07-4030-8dd4-1ce20656b7d2 admin admin] PATH_INFO: `/v3/projects/ea72b8902da644f39a2fa31441a7152c` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:49.972650 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137eb68a-de7d-4bc8-b06e-e16f03e80b8c admin admin] PATH_INFO: `/v3/users/d88a987ae16e466c88478eef0bcb96e9` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:49.972732 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ed1304-8e07-4030-8dd4-1ce20656b7d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea72b8902da644f39a2fa31441a7152c)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:49.972732 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ed1304-8e07-4030-8dd4-1ce20656b7d2 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:49.975465 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137eb68a-de7d-4bc8-b06e-e16f03e80b8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d88a987ae16e466c88478eef0bcb96e9)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:49.977561 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137eb68a-de7d-4bc8-b06e-e16f03e80b8c admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:50.165947 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-137eb68a-de7d-4bc8-b06e-e16f03e80b8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd88a987ae16e466c88478eef0bcb96e9'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:50.166113 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-49ed1304-8e07-4030-8dd4-1ce20656b7d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea72b8902da644f39a2fa31441a7152c', 'request_id': 'req-49ed1304-8e07-4030-8dd4-1ce20656b7d2'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:50.173199 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-137eb68a-de7d-4bc8-b06e-e16f03e80b8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd88a987ae16e466c88478eef0bcb96e9'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:50.173372 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-49ed1304-8e07-4030-8dd4-1ce20656b7d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea72b8902da644f39a2fa31441a7152c', 'request_id': 'req-49ed1304-8e07-4030-8dd4-1ce20656b7d2'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:50.173927 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-137eb68a-de7d-4bc8-b06e-e16f03e80b8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd88a987ae16e466c88478eef0bcb96e9'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:50.186630 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-137eb68a-de7d-4bc8-b06e-e16f03e80b8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd88a987ae16e466c88478eef0bcb96e9'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:50.225896 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-49ed1304-8e07-4030-8dd4-1ce20656b7d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea72b8902da644f39a2fa31441a7152c', 'request_id': 'req-49ed1304-8e07-4030-8dd4-1ce20656b7d2'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:50.237215 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-49ed1304-8e07-4030-8dd4-1ce20656b7d2 admin admin] The token cache is being invalidate because project ea72b8902da644f39a2fa31441a7152c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:13:50.237504 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-49ed1304-8e07-4030-8dd4-1ce20656b7d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:50.238154 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-49ed1304-8e07-4030-8dd4-1ce20656b7d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:50.239770 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 925/1882] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:13:49 2022] DELETE /identity/v3/projects/ea72b8902da644f39a2fa31441a7152c => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:50.240548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ed1304-8e07-4030-8dd4-1ce20656b7d2 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:50.242073 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-49ed1304-8e07-4030-8dd4-1ce20656b7d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:50.299856 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-137eb68a-de7d-4bc8-b06e-e16f03e80b8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd88a987ae16e466c88478eef0bcb96e9'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:50.306357 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ed1304-8e07-4030-8dd4-1ce20656b7d2 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:50.307046 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 958/1883] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:13:49 2022] DELETE /identity/v3/users/d88a987ae16e466c88478eef0bcb96e9 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:50.310102 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137eb68a-de7d-4bc8-b06e-e16f03e80b8c admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:50.310102 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-137eb68a-de7d-4bc8-b06e-e16f03e80b8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:50.369381 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e31e9ca-a5b2-441e-86f0-8b5b53a40590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:50.370277 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31e9ca-a5b2-441e-86f0-8b5b53a40590 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:50.370556 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31e9ca-a5b2-441e-86f0-8b5b53a40590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:50.370829 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31e9ca-a5b2-441e-86f0-8b5b53a40590 admin admin] PATH_INFO: `/v3/projects/2bffae38482b47519d492494d213aaab/users/af120e7625954132802a88898f48233a/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:50.375819 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137eb68a-de7d-4bc8-b06e-e16f03e80b8c admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:50.380573 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11b13d1-200d-4135-8fed-a98bc6390dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:50.381491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b13d1-200d-4135-8fed-a98bc6390dbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:50.381760 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b13d1-200d-4135-8fed-a98bc6390dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:50.382158 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11b13d1-200d-4135-8fed-a98bc6390dbe admin admin] PATH_INFO: `/v3/projects/0716872dd0514186bede1295d8c40031/users/8353ce80989c42fa866c64997435d6f5/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:50.385695 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e31e9ca-a5b2-441e-86f0-8b5b53a40590 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bffae38482b47519d492494d213aaab, user_id=af120e7625954132802a88898f48233a, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:50.388370 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e31e9ca-a5b2-441e-86f0-8b5b53a40590 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:50.399510 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11b13d1-200d-4135-8fed-a98bc6390dbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=0716872dd0514186bede1295d8c40031, user_id=8353ce80989c42fa866c64997435d6f5, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:50.402760 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11b13d1-200d-4135-8fed-a98bc6390dbe admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:50.426413 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:50.466313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 926/1884] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:50 2022] PUT /identity/v3/projects/2bffae38482b47519d492494d213aaab/users/af120e7625954132802a88898f48233a/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:50.467876 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe77e7b-a518-4504-a695-86c52c5509e3 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:50.468171 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe77e7b-a518-4504-a695-86c52c5509e3 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:50.468452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe77e7b-a518-4504-a695-86c52c5509e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:50.487924 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-5fe77e7b-a518-4504-a695-86c52c5509e3 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:50.489123 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-5fe77e7b-a518-4504-a695-86c52c5509e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:50.507624 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 959/1885] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:50 2022] PUT /identity/v3/projects/0716872dd0514186bede1295d8c40031/users/8353ce80989c42fa866c64997435d6f5/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:50.511881 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29edbab-575e-4bd3-995c-acaae9cf8303 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:50.512185 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29edbab-575e-4bd3-995c-acaae9cf8303 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:50.512475 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29edbab-575e-4bd3-995c-acaae9cf8303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:50.532448 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-d29edbab-575e-4bd3-995c-acaae9cf8303 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:50.533863 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d29edbab-575e-4bd3-995c-acaae9cf8303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:50.563762 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 927/1886] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:50.563762 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe77e7b-a518-4504-a695-86c52c5509e3 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:50.563762 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-5fe77e7b-a518-4504-a695-86c52c5509e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:50.575892 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe77e7b-a518-4504-a695-86c52c5509e3 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:50.594758 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 960/1887] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:50.595496 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29edbab-575e-4bd3-995c-acaae9cf8303 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:50.605084 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29edbab-575e-4bd3-995c-acaae9cf8303 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:50.612683 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac346a1-6cf8-455a-a6e3-68c746538fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:50.612683 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac346a1-6cf8-455a-a6e3-68c746538fa9 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:50.612683 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac346a1-6cf8-455a-a6e3-68c746538fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:50.612683 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac346a1-6cf8-455a-a6e3-68c746538fa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:50.613055 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac346a1-6cf8-455a-a6e3-68c746538fa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:50.613879 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac346a1-6cf8-455a-a6e3-68c746538fa9 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:50.627158 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 961/1888] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:50.627158 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e1e05-8b0b-4f65-8352-b64b87567447 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:50.627158 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e1e05-8b0b-4f65-8352-b64b87567447 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:50.627158 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8e1e05-8b0b-4f65-8352-b64b87567447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:50.632889 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a61ddf-9018-4f09-8bfa-3ad27e6d8a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:50.633593 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a61ddf-9018-4f09-8bfa-3ad27e6d8a3c admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:50.633876 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a61ddf-9018-4f09-8bfa-3ad27e6d8a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:50.634149 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a61ddf-9018-4f09-8bfa-3ad27e6d8a3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:50.634874 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a61ddf-9018-4f09-8bfa-3ad27e6d8a3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:50.636516 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a61ddf-9018-4f09-8bfa-3ad27e6d8a3c admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:50.644554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-6c8e1e05-8b0b-4f65-8352-b64b87567447 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:50.645778 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-6c8e1e05-8b0b-4f65-8352-b64b87567447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:50.647225 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 928/1889] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:50.648072 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a61ddf-9018-4f09-8bfa-3ad27e6d8a3c admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:50.649726 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-32a61ddf-9018-4f09-8bfa-3ad27e6d8a3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:50.654355 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 962/1890] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:50.655185 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8e1e05-8b0b-4f65-8352-b64b87567447 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:50.664580 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8e1e05-8b0b-4f65-8352-b64b87567447 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:50.669304 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e85f9a-10bb-4383-b467-f6aa026d1c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:50.670315 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e85f9a-10bb-4383-b467-f6aa026d1c85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:50.670607 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e85f9a-10bb-4383-b467-f6aa026d1c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:50.670914 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e85f9a-10bb-4383-b467-f6aa026d1c85 admin admin] PATH_INFO: `/v3/projects/0716872dd0514186bede1295d8c40031/users/8353ce80989c42fa866c64997435d6f5/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:50.672936 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e85f9a-10bb-4383-b467-f6aa026d1c85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0716872dd0514186bede1295d8c40031, user_id=8353ce80989c42fa866c64997435d6f5, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:50.675604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e85f9a-10bb-4383-b467-f6aa026d1c85 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:50.687321 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 963/1891] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:50 2022] PUT /identity/v3/projects/0716872dd0514186bede1295d8c40031/users/8353ce80989c42fa866c64997435d6f5/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:50.687878 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e85f9a-10bb-4383-b467-f6aa026d1c85 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:50.708343 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a61ddf-9018-4f09-8bfa-3ad27e6d8a3c admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:50.760324 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26362534-504e-4b55-b317-adca379f18ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:50.760988 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26362534-504e-4b55-b317-adca379f18ee service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:50.761092 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e85f9a-10bb-4383-b467-f6aa026d1c85 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:50.761333 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26362534-504e-4b55-b317-adca379f18ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:50.761619 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26362534-504e-4b55-b317-adca379f18ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:50.766217 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0275d8b-16f9-4b07-b8c6-66e2cb843300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:50.767104 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0275d8b-16f9-4b07-b8c6-66e2cb843300 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:50.767392 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0275d8b-16f9-4b07-b8c6-66e2cb843300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:50.768659 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0275d8b-16f9-4b07-b8c6-66e2cb843300 admin admin] PATH_INFO: `/v3/projects/2bffae38482b47519d492494d213aaab/users/af120e7625954132802a88898f48233a/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:50.769707 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0275d8b-16f9-4b07-b8c6-66e2cb843300 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bffae38482b47519d492494d213aaab, user_id=af120e7625954132802a88898f48233a, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:50.771872 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26362534-504e-4b55-b317-adca379f18ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:50.772415 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0275d8b-16f9-4b07-b8c6-66e2cb843300 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:50.773407 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26362534-504e-4b55-b317-adca379f18ee service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:50.783248 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 964/1892] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:50 2022] PUT /identity/v3/projects/2bffae38482b47519d492494d213aaab/users/af120e7625954132802a88898f48233a/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:50.784395 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0275d8b-16f9-4b07-b8c6-66e2cb843300 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:50.784512 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 929/1893] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:13:50.785964 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cd5cfe-c064-443c-9923-0ac7e0c96c95 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:50.786335 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cd5cfe-c064-443c-9923-0ac7e0c96c95 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:50.786625 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cd5cfe-c064-443c-9923-0ac7e0c96c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:50.819611 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-40cd5cfe-c064-443c-9923-0ac7e0c96c95 None None] MFA Rules not processed for user `8353ce80989c42fa866c64997435d6f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:50.820927 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-40cd5cfe-c064-443c-9923-0ac7e0c96c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:50.858060 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0275d8b-16f9-4b07-b8c6-66e2cb843300 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:50.862956 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f354f762-7f92-4e6c-935d-371e9792d678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:50.863633 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354f762-7f92-4e6c-935d-371e9792d678 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:50.863970 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354f762-7f92-4e6c-935d-371e9792d678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:50.864236 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354f762-7f92-4e6c-935d-371e9792d678 admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:50.867602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f354f762-7f92-4e6c-935d-371e9792d678 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:13:50.867602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f354f762-7f92-4e6c-935d-371e9792d678 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:50.867602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f354f762-7f92-4e6c-935d-371e9792d678 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:50.879790 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 965/1894] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:13:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:13:50.881286 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff2af6-2ea7-46b3-8fa6-e7041c842d61 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:50.881705 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff2af6-2ea7-46b3-8fa6-e7041c842d61 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:50.881996 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff2af6-2ea7-46b3-8fa6-e7041c842d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:50.896638 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 930/1895] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:50 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:50.897572 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cd5cfe-c064-443c-9923-0ac7e0c96c95 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:50.909294 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cd5cfe-c064-443c-9923-0ac7e0c96c95 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:50.913546 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-7fff2af6-2ea7-46b3-8fa6-e7041c842d61 None None] MFA Rules not processed for user `af120e7625954132802a88898f48233a`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:50.913960 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e0f8dd-055c-4b24-aaf8-9e1a843e34e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:50.914621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e0f8dd-055c-4b24-aaf8-9e1a843e34e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:50.914809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-7fff2af6-2ea7-46b3-8fa6-e7041c842d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:50.914906 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e0f8dd-055c-4b24-aaf8-9e1a843e34e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:50.915175 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e0f8dd-055c-4b24-aaf8-9e1a843e34e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:50.915898 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e0f8dd-055c-4b24-aaf8-9e1a843e34e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:50.917809 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e0f8dd-055c-4b24-aaf8-9e1a843e34e1 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:50.982138 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 966/1896] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:50 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:50.983157 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff2af6-2ea7-46b3-8fa6-e7041c842d61 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:50.984855 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-7fff2af6-2ea7-46b3-8fa6-e7041c842d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:51.022669 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 931/1897] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:13:50 2022] POST /identity/v3/projects => generated 379 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:51.022669 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e0f8dd-055c-4b24-aaf8-9e1a843e34e1 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:51.022669 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-20e0f8dd-055c-4b24-aaf8-9e1a843e34e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:51.064968 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff2af6-2ea7-46b3-8fa6-e7041c842d61 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:51.093365 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e0f8dd-055c-4b24-aaf8-9e1a843e34e1 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:51.097812 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6714c0-a735-4b5d-ac9f-f3356f76a7a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:51.098644 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6714c0-a735-4b5d-ac9f-f3356f76a7a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:51.098928 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6714c0-a735-4b5d-ac9f-f3356f76a7a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:51.099201 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6714c0-a735-4b5d-ac9f-f3356f76a7a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:51.101473 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-9a6714c0-a735-4b5d-ac9f-f3356f76a7a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:51.103389 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:51.131543 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c75d89b-11e6-44e4-bf27-932ba4cf8053 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:51.132259 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75d89b-11e6-44e4-bf27-932ba4cf8053 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:51.133687 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75d89b-11e6-44e4-bf27-932ba4cf8053 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:51.133687 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c75d89b-11e6-44e4-bf27-932ba4cf8053 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:51.135937 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-4c75d89b-11e6-44e4-bf27-932ba4cf8053 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:51.180736 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6714c0-a735-4b5d-ac9f-f3356f76a7a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:51.182362 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6714c0-a735-4b5d-ac9f-f3356f76a7a1 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:51.196804 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 932/1898] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:13:51.196804 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6714c0-a735-4b5d-ac9f-f3356f76a7a1 service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:51.216768 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c75d89b-11e6-44e4-bf27-932ba4cf8053 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:51.218892 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c75d89b-11e6-44e4-bf27-932ba4cf8053 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:51.229562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 967/1899] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:13:51.278373 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6714c0-a735-4b5d-ac9f-f3356f76a7a1 service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:51.283833 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d7beb-1c47-42c9-9a03-8d5faf4e4086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:51.284574 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d7beb-1c47-42c9-9a03-8d5faf4e4086 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:51.284859 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d7beb-1c47-42c9-9a03-8d5faf4e4086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:51.285272 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d7beb-1c47-42c9-9a03-8d5faf4e4086 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:51.285932 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d7beb-1c47-42c9-9a03-8d5faf4e4086 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:51.291804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c75d89b-11e6-44e4-bf27-932ba4cf8053 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:51.293523 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-4c75d89b-11e6-44e4-bf27-932ba4cf8053 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:51.297665 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d7beb-1c47-42c9-9a03-8d5faf4e4086 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:51.318228 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c75d89b-11e6-44e4-bf27-932ba4cf8053 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:51.329183 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4984f7-01e3-4d57-965c-ae2a5e8be501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:51.330254 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4984f7-01e3-4d57-965c-ae2a5e8be501 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:51.330601 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4984f7-01e3-4d57-965c-ae2a5e8be501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:51.330985 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4984f7-01e3-4d57-965c-ae2a5e8be501 admin admin] PATH_INFO: `/v3/projects/b7f052cc276747b9bc9acbf22ccbfa1f` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:51.332362 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4984f7-01e3-4d57-965c-ae2a5e8be501 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7f052cc276747b9bc9acbf22ccbfa1f)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:51.334522 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4984f7-01e3-4d57-965c-ae2a5e8be501 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:51.398205 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 933/1900] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:13:51 2022] POST /identity/v3/users => generated 290 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:51.402739 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d7beb-1c47-42c9-9a03-8d5faf4e4086 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:51.416947 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d7beb-1c47-42c9-9a03-8d5faf4e4086 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:51.437371 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e67799f-0e80-4f16-82e5-4bf7b26834b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:51.438066 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67799f-0e80-4f16-82e5-4bf7b26834b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:51.438378 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67799f-0e80-4f16-82e5-4bf7b26834b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:51.438692 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67799f-0e80-4f16-82e5-4bf7b26834b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:51.439583 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e67799f-0e80-4f16-82e5-4bf7b26834b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:51.441200 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e67799f-0e80-4f16-82e5-4bf7b26834b6 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:51.451981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 934/1901] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:13:51.456025 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e67799f-0e80-4f16-82e5-4bf7b26834b6 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:51.468388 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e67799f-0e80-4f16-82e5-4bf7b26834b6 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:51.474427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2036f505-44d3-4a6c-a44e-dcad384392d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:51.475296 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036f505-44d3-4a6c-a44e-dcad384392d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:51.475804 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036f505-44d3-4a6c-a44e-dcad384392d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:51.476135 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2036f505-44d3-4a6c-a44e-dcad384392d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:51.477288 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2036f505-44d3-4a6c-a44e-dcad384392d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:51.479177 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2036f505-44d3-4a6c-a44e-dcad384392d4 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:51.493577 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 935/1902] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:13:51.498223 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2036f505-44d3-4a6c-a44e-dcad384392d4 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:51.519637 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2036f505-44d3-4a6c-a44e-dcad384392d4 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:51.539367 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8d4984f7-01e3-4d57-965c-ae2a5e8be501 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7f052cc276747b9bc9acbf22ccbfa1f', 'request_id': 'req-8d4984f7-01e3-4d57-965c-ae2a5e8be501'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:51.546586 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8d4984f7-01e3-4d57-965c-ae2a5e8be501 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7f052cc276747b9bc9acbf22ccbfa1f', 'request_id': 'req-8d4984f7-01e3-4d57-965c-ae2a5e8be501'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:51.553951 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8d4984f7-01e3-4d57-965c-ae2a5e8be501 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7f052cc276747b9bc9acbf22ccbfa1f', 'request_id': 'req-8d4984f7-01e3-4d57-965c-ae2a5e8be501'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:51.605046 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96973de0-1402-450c-98d6-257913a17236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:51.606386 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96973de0-1402-450c-98d6-257913a17236 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:51.606675 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96973de0-1402-450c-98d6-257913a17236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:51.606958 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96973de0-1402-450c-98d6-257913a17236 admin admin] PATH_INFO: `/v3/projects/8901313d60ea42378d4e1c2d8dfe40a9/users/2321fd2dec8141eba5a3b03a9af41790/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:51.636841 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8d4984f7-01e3-4d57-965c-ae2a5e8be501 admin admin] The token cache is being invalidate because project b7f052cc276747b9bc9acbf22ccbfa1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:13:51.637147 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8d4984f7-01e3-4d57-965c-ae2a5e8be501 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:51.640228 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8d4984f7-01e3-4d57-965c-ae2a5e8be501 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:51.643603 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 968/1903] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:13:51 2022] DELETE /identity/v3/projects/b7f052cc276747b9bc9acbf22ccbfa1f => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:13:51.648394 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4984f7-01e3-4d57-965c-ae2a5e8be501 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:51.649832 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-8d4984f7-01e3-4d57-965c-ae2a5e8be501 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:51.658620 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96973de0-1402-450c-98d6-257913a17236 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8901313d60ea42378d4e1c2d8dfe40a9, user_id=2321fd2dec8141eba5a3b03a9af41790, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:51.661752 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96973de0-1402-450c-98d6-257913a17236 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:51.672318 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4984f7-01e3-4d57-965c-ae2a5e8be501 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:51.677794 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ad61b-5b54-4ea8-a4a4-6e0764ac183e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:51.678766 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ad61b-5b54-4ea8-a4a4-6e0764ac183e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:51.679061 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ad61b-5b54-4ea8-a4a4-6e0764ac183e admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:51.679459 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1ad61b-5b54-4ea8-a4a4-6e0764ac183e admin admin] PATH_INFO: `/v3/users/4747c69e14704d03a7a1dc51a342059e` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:51.681440 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ad61b-5b54-4ea8-a4a4-6e0764ac183e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4747c69e14704d03a7a1dc51a342059e)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:51.683684 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1ad61b-5b54-4ea8-a4a4-6e0764ac183e admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:51.752912 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 936/1904] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:51 2022] PUT /identity/v3/projects/8901313d60ea42378d4e1c2d8dfe40a9/users/2321fd2dec8141eba5a3b03a9af41790/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:51.764932 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003d9c4-f955-45b1-a4f0-0c830020239b None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:51.765534 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003d9c4-f955-45b1-a4f0-0c830020239b None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:51.765656 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003d9c4-f955-45b1-a4f0-0c830020239b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:51.794507 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-8003d9c4-f955-45b1-a4f0-0c830020239b None None] MFA Rules not processed for user `2321fd2dec8141eba5a3b03a9af41790`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:51.796756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-8003d9c4-f955-45b1-a4f0-0c830020239b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:51.833827 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:51.867837 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 937/1905] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:51 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:13:51.872732 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8003d9c4-f955-45b1-a4f0-0c830020239b None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:51.874784 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-8003d9c4-f955-45b1-a4f0-0c830020239b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:51.929538 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-6e1ad61b-5b54-4ea8-a4a4-6e0764ac183e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4747c69e14704d03a7a1dc51a342059e'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:51.939593 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-6e1ad61b-5b54-4ea8-a4a4-6e0764ac183e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4747c69e14704d03a7a1dc51a342059e'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:51.942285 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-6e1ad61b-5b54-4ea8-a4a4-6e0764ac183e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4747c69e14704d03a7a1dc51a342059e'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:51.959437 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-6e1ad61b-5b54-4ea8-a4a4-6e0764ac183e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4747c69e14704d03a7a1dc51a342059e'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:51.961384 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8003d9c4-f955-45b1-a4f0-0c830020239b None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:51.968414 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b1aeb-0fca-46e6-a10d-96cbed5b3963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:51.969213 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b1aeb-0fca-46e6-a10d-96cbed5b3963 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:51.969575 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b1aeb-0fca-46e6-a10d-96cbed5b3963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:51.970208 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b1aeb-0fca-46e6-a10d-96cbed5b3963 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:51.970758 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53b1aeb-0fca-46e6-a10d-96cbed5b3963 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:51.973452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53b1aeb-0fca-46e6-a10d-96cbed5b3963 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:52.054147 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-6e1ad61b-5b54-4ea8-a4a4-6e0764ac183e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4747c69e14704d03a7a1dc51a342059e'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:52.064497 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 969/1906] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:13:51 2022] DELETE /identity/v3/users/4747c69e14704d03a7a1dc51a342059e => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:13:52.091980 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 938/1907] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:13:51 2022] POST /identity/v3/projects => generated 379 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:52.096635 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ad61b-5b54-4ea8-a4a4-6e0764ac183e admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:52.098573 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-6e1ad61b-5b54-4ea8-a4a4-6e0764ac183e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:52.217086 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1ad61b-5b54-4ea8-a4a4-6e0764ac183e admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:52.279623 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2058174-63a3-40b7-9928-d106b9c091c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:52.281342 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2058174-63a3-40b7-9928-d106b9c091c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:52.281775 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2058174-63a3-40b7-9928-d106b9c091c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:52.282515 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2058174-63a3-40b7-9928-d106b9c091c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:52.282767 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2058174-63a3-40b7-9928-d106b9c091c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:52.284770 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2058174-63a3-40b7-9928-d106b9c091c6 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:52.410878 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 970/1908] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:13:52 2022] POST /identity/v3/users => generated 296 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:52.412032 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b1aeb-0fca-46e6-a10d-96cbed5b3963 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:52.426340 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b1aeb-0fca-46e6-a10d-96cbed5b3963 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:52.434857 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c3af0-5257-48dc-ab1f-ffeb8a94ed99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:52.435678 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c3af0-5257-48dc-ab1f-ffeb8a94ed99 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:52.436160 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c3af0-5257-48dc-ab1f-ffeb8a94ed99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:52.436540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c3af0-5257-48dc-ab1f-ffeb8a94ed99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:52.437721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63c3af0-5257-48dc-ab1f-ffeb8a94ed99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:52.439854 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63c3af0-5257-48dc-ab1f-ffeb8a94ed99 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:52.459030 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 939/1909] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:52.459822 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2058174-63a3-40b7-9928-d106b9c091c6 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:52.474960 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2058174-63a3-40b7-9928-d106b9c091c6 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:52.487390 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4b4ee-8481-4c74-9f81-1c9af1c05c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:52.490147 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4b4ee-8481-4c74-9f81-1c9af1c05c36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:52.490646 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4b4ee-8481-4c74-9f81-1c9af1c05c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:52.491147 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4b4ee-8481-4c74-9f81-1c9af1c05c36 admin admin] PATH_INFO: `/v3/projects/c2ba1de3f391450aa4e744068d800fbd/users/71a93e1f76dc427d906c3ed4bd8249ce/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:52.510703 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d4b4ee-8481-4c74-9f81-1c9af1c05c36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2ba1de3f391450aa4e744068d800fbd, user_id=71a93e1f76dc427d906c3ed4bd8249ce, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:52.516922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d4b4ee-8481-4c74-9f81-1c9af1c05c36 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:52.599838 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 971/1910] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:52 2022] PUT /identity/v3/projects/c2ba1de3f391450aa4e744068d800fbd/users/71a93e1f76dc427d906c3ed4bd8249ce/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:13:52.603977 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c3af0-5257-48dc-ab1f-ffeb8a94ed99 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:52.616452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c3af0-5257-48dc-ab1f-ffeb8a94ed99 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:52.688439 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f9365-02c4-4cae-b2c8-ca48b4ebfc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:52.689154 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f9365-02c4-4cae-b2c8-ca48b4ebfc7a admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:52.689439 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f9365-02c4-4cae-b2c8-ca48b4ebfc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:52.689714 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f9365-02c4-4cae-b2c8-ca48b4ebfc7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:52.690444 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565f9365-02c4-4cae-b2c8-ca48b4ebfc7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:52.692085 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565f9365-02c4-4cae-b2c8-ca48b4ebfc7a admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:52.706132 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 940/1911] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:52.713621 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4b4ee-8481-4c74-9f81-1c9af1c05c36 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:52.741369 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4b4ee-8481-4c74-9f81-1c9af1c05c36 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:52.746831 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f740273-3703-4c31-9065-5c0bea0e4e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:52.750627 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f740273-3703-4c31-9065-5c0bea0e4e2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:52.750627 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f740273-3703-4c31-9065-5c0bea0e4e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:52.750627 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f740273-3703-4c31-9065-5c0bea0e4e2e admin admin] PATH_INFO: `/v3/projects/c2ba1de3f391450aa4e744068d800fbd/users/71a93e1f76dc427d906c3ed4bd8249ce/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:52.751103 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f740273-3703-4c31-9065-5c0bea0e4e2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2ba1de3f391450aa4e744068d800fbd, user_id=71a93e1f76dc427d906c3ed4bd8249ce, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:52.754192 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f740273-3703-4c31-9065-5c0bea0e4e2e admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:52.773454 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 972/1912] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:52 2022] PUT /identity/v3/projects/c2ba1de3f391450aa4e744068d800fbd/users/71a93e1f76dc427d906c3ed4bd8249ce/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:13:52.783870 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73f779-b331-4653-a8b7-78211052b0f6 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:52.784817 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73f779-b331-4653-a8b7-78211052b0f6 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:52.785250 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac73f779-b331-4653-a8b7-78211052b0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:52.811924 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f740273-3703-4c31-9065-5c0bea0e4e2e admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:52.813867 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-3f740273-3703-4c31-9065-5c0bea0e4e2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:52.858288 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-ac73f779-b331-4653-a8b7-78211052b0f6 None None] MFA Rules not processed for user `71a93e1f76dc427d906c3ed4bd8249ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:52.860265 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ac73f779-b331-4653-a8b7-78211052b0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:52.932636 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f740273-3703-4c31-9065-5c0bea0e4e2e admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:52.943218 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12804d2-4243-45e2-b04a-cbb7ebab408d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:52.944658 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12804d2-4243-45e2-b04a-cbb7ebab408d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:52.947129 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12804d2-4243-45e2-b04a-cbb7ebab408d admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:52.947592 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12804d2-4243-45e2-b04a-cbb7ebab408d admin admin] PATH_INFO: `/v3/projects/973d3ba0ed7b40c4b80ca6f685ece74b` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:52.950867 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12804d2-4243-45e2-b04a-cbb7ebab408d admin admin] RBAC: Authorizing `identity:delete_project(project_id=973d3ba0ed7b40c4b80ca6f685ece74b)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:52.954071 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12804d2-4243-45e2-b04a-cbb7ebab408d admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:53.023435 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 941/1913] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:52 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:53.037922 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac73f779-b331-4653-a8b7-78211052b0f6 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:53.047630 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ac73f779-b331-4653-a8b7-78211052b0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:53.148469 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac73f779-b331-4653-a8b7-78211052b0f6 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:53.151078 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f12804d2-4243-45e2-b04a-cbb7ebab408d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '973d3ba0ed7b40c4b80ca6f685ece74b', 'request_id': 'req-f12804d2-4243-45e2-b04a-cbb7ebab408d'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:53.160410 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f12804d2-4243-45e2-b04a-cbb7ebab408d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '973d3ba0ed7b40c4b80ca6f685ece74b', 'request_id': 'req-f12804d2-4243-45e2-b04a-cbb7ebab408d'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:53.170719 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f12804d2-4243-45e2-b04a-cbb7ebab408d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '973d3ba0ed7b40c4b80ca6f685ece74b', 'request_id': 'req-f12804d2-4243-45e2-b04a-cbb7ebab408d'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:53.209441 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:53.247759 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f12804d2-4243-45e2-b04a-cbb7ebab408d admin admin] The token cache is being invalidate because project 973d3ba0ed7b40c4b80ca6f685ece74b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:13:53.248075 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f12804d2-4243-45e2-b04a-cbb7ebab408d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:53.249388 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f12804d2-4243-45e2-b04a-cbb7ebab408d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:53.251026 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e10b1c5-12ed-43d2-9c01-dd6a2b43a5e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:53.252689 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e10b1c5-12ed-43d2-9c01-dd6a2b43a5e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:53.253386 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e10b1c5-12ed-43d2-9c01-dd6a2b43a5e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:53.254010 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e10b1c5-12ed-43d2-9c01-dd6a2b43a5e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:53.256113 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 973/1914] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:13:52 2022] DELETE /identity/v3/projects/973d3ba0ed7b40c4b80ca6f685ece74b => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:53.259311 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12804d2-4243-45e2-b04a-cbb7ebab408d admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:53.261452 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f12804d2-4243-45e2-b04a-cbb7ebab408d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:53.266989 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-7e10b1c5-12ed-43d2-9c01-dd6a2b43a5e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:53.369139 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12804d2-4243-45e2-b04a-cbb7ebab408d admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:53.378294 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0173fc-cbe7-47cc-b259-114a2d689ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:53.379664 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0173fc-cbe7-47cc-b259-114a2d689ec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:53.380561 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0173fc-cbe7-47cc-b259-114a2d689ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:53.380852 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0173fc-cbe7-47cc-b259-114a2d689ec9 admin admin] PATH_INFO: `/v3/users/2a0c77cb5a0d42088f281b1f790fe22c` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:53.382462 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0173fc-cbe7-47cc-b259-114a2d689ec9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a0c77cb5a0d42088f281b1f790fe22c)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:53.386444 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0173fc-cbe7-47cc-b259-114a2d689ec9 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:53.395844 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:53.449494 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e10b1c5-12ed-43d2-9c01-dd6a2b43a5e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:53.450971 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e10b1c5-12ed-43d2-9c01-dd6a2b43a5e6 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:53.467760 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 942/1915] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:53 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 17:13:53.650217 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-df0173fc-cbe7-47cc-b259-114a2d689ec9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a0c77cb5a0d42088f281b1f790fe22c'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:53.660171 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-df0173fc-cbe7-47cc-b259-114a2d689ec9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a0c77cb5a0d42088f281b1f790fe22c'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:53.660948 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-df0173fc-cbe7-47cc-b259-114a2d689ec9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a0c77cb5a0d42088f281b1f790fe22c'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:53.673504 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-df0173fc-cbe7-47cc-b259-114a2d689ec9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a0c77cb5a0d42088f281b1f790fe22c'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:53.783627 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-df0173fc-cbe7-47cc-b259-114a2d689ec9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a0c77cb5a0d42088f281b1f790fe22c'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:53.790175 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 974/1916] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:13:53 2022] DELETE /identity/v3/users/2a0c77cb5a0d42088f281b1f790fe22c => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:53.883563 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e10b1c5-12ed-43d2-9c01-dd6a2b43a5e6 service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:53.886198 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-7e10b1c5-12ed-43d2-9c01-dd6a2b43a5e6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:53.897065 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e10b1c5-12ed-43d2-9c01-dd6a2b43a5e6 service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:53.949416 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4703770d-13d6-4144-980e-07cf219374d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:53.950130 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703770d-13d6-4144-980e-07cf219374d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:53.950420 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703770d-13d6-4144-980e-07cf219374d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:53.950700 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4703770d-13d6-4144-980e-07cf219374d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:53.952595 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4703770d-13d6-4144-980e-07cf219374d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:54.056406 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4703770d-13d6-4144-980e-07cf219374d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:54.058053 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4703770d-13d6-4144-980e-07cf219374d5 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:54.073078 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 943/1917] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:13:54.362785 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0173fc-cbe7-47cc-b259-114a2d689ec9 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:54.364536 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-df0173fc-cbe7-47cc-b259-114a2d689ec9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:54.466553 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0173fc-cbe7-47cc-b259-114a2d689ec9 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:54.471721 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203d89f-9ae5-4f29-b1e0-4d59114c08aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:54.472613 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203d89f-9ae5-4f29-b1e0-4d59114c08aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:54.472897 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203d89f-9ae5-4f29-b1e0-4d59114c08aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:54.473172 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203d89f-9ae5-4f29-b1e0-4d59114c08aa admin admin] PATH_INFO: `/v3/projects/45e9dbd4612b4e209e89664c3862700e` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:54.474345 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a203d89f-9ae5-4f29-b1e0-4d59114c08aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=45e9dbd4612b4e209e89664c3862700e)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:54.476322 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a203d89f-9ae5-4f29-b1e0-4d59114c08aa admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:54.581134 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-a203d89f-9ae5-4f29-b1e0-4d59114c08aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45e9dbd4612b4e209e89664c3862700e', 'request_id': 'req-a203d89f-9ae5-4f29-b1e0-4d59114c08aa'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:54.587225 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-a203d89f-9ae5-4f29-b1e0-4d59114c08aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45e9dbd4612b4e209e89664c3862700e', 'request_id': 'req-a203d89f-9ae5-4f29-b1e0-4d59114c08aa'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:54.592830 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-a203d89f-9ae5-4f29-b1e0-4d59114c08aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45e9dbd4612b4e209e89664c3862700e', 'request_id': 'req-a203d89f-9ae5-4f29-b1e0-4d59114c08aa'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:54.646448 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4703770d-13d6-4144-980e-07cf219374d5 service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:54.647958 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4703770d-13d6-4144-980e-07cf219374d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:54.661481 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4703770d-13d6-4144-980e-07cf219374d5 service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:54.664663 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-a203d89f-9ae5-4f29-b1e0-4d59114c08aa admin admin] The token cache is being invalidate because project 45e9dbd4612b4e209e89664c3862700e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:13:54.664958 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-a203d89f-9ae5-4f29-b1e0-4d59114c08aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:54.668246 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-a203d89f-9ae5-4f29-b1e0-4d59114c08aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:54.668246 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 975/1918] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:13:54 2022] DELETE /identity/v3/projects/45e9dbd4612b4e209e89664c3862700e => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:54.673122 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52841aa-3db0-489c-b1e5-2b3660054315 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:54.673417 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52841aa-3db0-489c-b1e5-2b3660054315 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:54.673710 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52841aa-3db0-489c-b1e5-2b3660054315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:54.693038 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-f52841aa-3db0-489c-b1e5-2b3660054315 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:54.694242 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f52841aa-3db0-489c-b1e5-2b3660054315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:54.720977 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a84d37-fb11-415a-895c-14f813a219ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:54.721907 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a84d37-fb11-415a-895c-14f813a219ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:54.722203 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a84d37-fb11-415a-895c-14f813a219ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:54.722465 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a84d37-fb11-415a-895c-14f813a219ec admin admin] PATH_INFO: `/v3/users/2321fd2dec8141eba5a3b03a9af41790` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:54.723678 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a84d37-fb11-415a-895c-14f813a219ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=2321fd2dec8141eba5a3b03a9af41790)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:54.726379 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a84d37-fb11-415a-895c-14f813a219ec admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:54.766119 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 976/1919] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:13:54.768447 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:54.775491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d134a8-0b97-4d83-9a94-e634f7ca344d None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:54.775818 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d134a8-0b97-4d83-9a94-e634f7ca344d None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:54.776102 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d134a8-0b97-4d83-9a94-e634f7ca344d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:54.795794 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-31d134a8-0b97-4d83-9a94-e634f7ca344d None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:54.797353 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-31d134a8-0b97-4d83-9a94-e634f7ca344d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:54.805968 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 977/1920] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:13:54.810316 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d134a8-0b97-4d83-9a94-e634f7ca344d None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:54.829567 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d134a8-0b97-4d83-9a94-e634f7ca344d None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:54.836054 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e8261-25b7-4f76-8d10-cc478df4a7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:54.838652 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e8261-25b7-4f76-8d10-cc478df4a7c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:54.838652 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e8261-25b7-4f76-8d10-cc478df4a7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:54.838652 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e8261-25b7-4f76-8d10-cc478df4a7c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:54.838652 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e8261-25b7-4f76-8d10-cc478df4a7c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:13:54.838652 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e8261-25b7-4f76-8d10-cc478df4a7c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:54.840130 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e8261-25b7-4f76-8d10-cc478df4a7c1 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:54.852811 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 978/1921] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:13:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:13:54.857120 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e8261-25b7-4f76-8d10-cc478df4a7c1 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:54.867887 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e8261-25b7-4f76-8d10-cc478df4a7c1 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:54.872527 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de7b117-6cc0-45c3-a8cc-a1f84edcbe55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:54.873792 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de7b117-6cc0-45c3-a8cc-a1f84edcbe55 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:54.874076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de7b117-6cc0-45c3-a8cc-a1f84edcbe55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:54.874353 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de7b117-6cc0-45c3-a8cc-a1f84edcbe55 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:54.875041 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de7b117-6cc0-45c3-a8cc-a1f84edcbe55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:54.877134 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de7b117-6cc0-45c3-a8cc-a1f84edcbe55 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:54.943998 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-a2a84d37-fb11-415a-895c-14f813a219ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2321fd2dec8141eba5a3b03a9af41790'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:54.956284 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-a2a84d37-fb11-415a-895c-14f813a219ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2321fd2dec8141eba5a3b03a9af41790'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:54.963765 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-a2a84d37-fb11-415a-895c-14f813a219ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2321fd2dec8141eba5a3b03a9af41790'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:54.964517 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 979/1922] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:13:54 2022] POST /identity/v3/projects => generated 389 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:13:54.964637 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-a2a84d37-fb11-415a-895c-14f813a219ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2321fd2dec8141eba5a3b03a9af41790'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:54.968522 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de7b117-6cc0-45c3-a8cc-a1f84edcbe55 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:55.029927 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de7b117-6cc0-45c3-a8cc-a1f84edcbe55 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:55.035079 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79228723-6e20-499f-97cc-934646be50ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:55.035768 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79228723-6e20-499f-97cc-934646be50ff admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:55.036156 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79228723-6e20-499f-97cc-934646be50ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:55.036604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79228723-6e20-499f-97cc-934646be50ff admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:55.037480 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79228723-6e20-499f-97cc-934646be50ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:55.039746 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79228723-6e20-499f-97cc-934646be50ff admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:55.042643 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-a2a84d37-fb11-415a-895c-14f813a219ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2321fd2dec8141eba5a3b03a9af41790'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:55.049712 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 944/1923] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:13:54 2022] DELETE /identity/v3/users/2321fd2dec8141eba5a3b03a9af41790 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:13:55.054301 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d68e5-d64e-4250-a5d5-ad631a84a958 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:55.054613 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d68e5-d64e-4250-a5d5-ad631a84a958 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:55.054890 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3d68e5-d64e-4250-a5d5-ad631a84a958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:55.072745 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-2d3d68e5-d64e-4250-a5d5-ad631a84a958 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:55.073947 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-2d3d68e5-d64e-4250-a5d5-ad631a84a958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:55.139240 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 945/1924] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:13:55.139737 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 980/1925] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:13:54 2022] POST /identity/v3/users => generated 295 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:13:55.143644 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3d68e5-d64e-4250-a5d5-ad631a84a958 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:55.146086 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:55.146925 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-2d3d68e5-d64e-4250-a5d5-ad631a84a958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:55.148946 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79228723-6e20-499f-97cc-934646be50ff admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:55.150706 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-79228723-6e20-499f-97cc-934646be50ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:55.159549 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3d68e5-d64e-4250-a5d5-ad631a84a958 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:55.164738 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ffa8e-5fa6-45cf-b620-a6b3bc8137f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:55.165530 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ffa8e-5fa6-45cf-b620-a6b3bc8137f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:55.167117 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ffa8e-5fa6-45cf-b620-a6b3bc8137f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:55.167117 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ffa8e-5fa6-45cf-b620-a6b3bc8137f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:55.167117 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5ffa8e-5fa6-45cf-b620-a6b3bc8137f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:55.168575 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5ffa8e-5fa6-45cf-b620-a6b3bc8137f6 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:55.179405 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 946/1926] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:13:55.183201 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ffa8e-5fa6-45cf-b620-a6b3bc8137f6 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:55.192801 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ffa8e-5fa6-45cf-b620-a6b3bc8137f6 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:55.197612 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505fe613-f092-45b3-8795-0975df5915eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:55.198266 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505fe613-f092-45b3-8795-0975df5915eb admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:55.198556 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505fe613-f092-45b3-8795-0975df5915eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:55.198837 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505fe613-f092-45b3-8795-0975df5915eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:55.199561 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505fe613-f092-45b3-8795-0975df5915eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:55.201308 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505fe613-f092-45b3-8795-0975df5915eb admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:55.208960 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79228723-6e20-499f-97cc-934646be50ff admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:55.211185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 947/1927] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:13:55.215278 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505fe613-f092-45b3-8795-0975df5915eb admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:55.225785 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505fe613-f092-45b3-8795-0975df5915eb admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:55.230823 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90599410-ac5b-4a5c-872d-44965dc63b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:55.231792 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90599410-ac5b-4a5c-872d-44965dc63b06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:55.232083 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90599410-ac5b-4a5c-872d-44965dc63b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:55.232365 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90599410-ac5b-4a5c-872d-44965dc63b06 admin admin] PATH_INFO: `/v3/projects/3c400aae4350471291c678e2e4972aff/users/c2b09349284d4c8f99876ae7277095b9/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:55.248803 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90599410-ac5b-4a5c-872d-44965dc63b06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c400aae4350471291c678e2e4972aff, user_id=c2b09349284d4c8f99876ae7277095b9, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:55.251463 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90599410-ac5b-4a5c-872d-44965dc63b06 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:55.261721 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cd814f-a75e-432d-81cb-8912d3b85c2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:55.262416 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd814f-a75e-432d-81cb-8912d3b85c2f service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:55.262661 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd814f-a75e-432d-81cb-8912d3b85c2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:55.263008 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cd814f-a75e-432d-81cb-8912d3b85c2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:55.274511 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cd814f-a75e-432d-81cb-8912d3b85c2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:55.276218 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cd814f-a75e-432d-81cb-8912d3b85c2f service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:55.287588 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 981/1928] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:13:55.320478 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 948/1929] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:55 2022] PUT /identity/v3/projects/3c400aae4350471291c678e2e4972aff/users/c2b09349284d4c8f99876ae7277095b9/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:13:55.332767 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced2f9d-cfab-49c9-84b5-8a2bfe83d07f None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:55.333055 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced2f9d-cfab-49c9-84b5-8a2bfe83d07f None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:55.333479 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ced2f9d-cfab-49c9-84b5-8a2bfe83d07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:55.363997 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-6ced2f9d-cfab-49c9-84b5-8a2bfe83d07f None None] MFA Rules not processed for user `c2b09349284d4c8f99876ae7277095b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:55.365221 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-6ced2f9d-cfab-49c9-84b5-8a2bfe83d07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:55.443678 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 982/1930] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:55.446684 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90599410-ac5b-4a5c-872d-44965dc63b06 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:55.516028 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90599410-ac5b-4a5c-872d-44965dc63b06 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:55.520869 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc962cf6-508a-43f1-beae-536144fc37b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:55.521539 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc962cf6-508a-43f1-beae-536144fc37b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:55.521825 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc962cf6-508a-43f1-beae-536144fc37b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:55.522107 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc962cf6-508a-43f1-beae-536144fc37b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:55.522896 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc962cf6-508a-43f1-beae-536144fc37b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:55.524885 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc962cf6-508a-43f1-beae-536144fc37b6 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:55.633697 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 949/1931] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:13:55 2022] POST /identity/v3/projects => generated 389 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:55.637906 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced2f9d-cfab-49c9-84b5-8a2bfe83d07f None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:55.648855 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ced2f9d-cfab-49c9-84b5-8a2bfe83d07f None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:55.710436 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb0cd6d-c5ab-408d-be0c-64510620ff2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:55.711125 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0cd6d-c5ab-408d-be0c-64510620ff2e admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:55.711412 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0cd6d-c5ab-408d-be0c-64510620ff2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:55.711694 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb0cd6d-c5ab-408d-be0c-64510620ff2e admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:55.712439 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb0cd6d-c5ab-408d-be0c-64510620ff2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:55.714389 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb0cd6d-c5ab-408d-be0c-64510620ff2e admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:55.815306 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 983/1932] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:13:55 2022] POST /identity/v3/users => generated 301 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:55.818281 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc962cf6-508a-43f1-beae-536144fc37b6 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:55.820304 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-dc962cf6-508a-43f1-beae-536144fc37b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:55.824826 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb0cd6d-c5ab-408d-be0c-64510620ff2e admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:55.834074 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc962cf6-508a-43f1-beae-536144fc37b6 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:55.838806 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb0cd6d-c5ab-408d-be0c-64510620ff2e admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:55.846647 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ec39b0-dbeb-429e-a904-866f615de2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:55.846647 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec39b0-dbeb-429e-a904-866f615de2c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:55.846647 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec39b0-dbeb-429e-a904-866f615de2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:55.846647 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ec39b0-dbeb-429e-a904-866f615de2c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:55.847536 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ec39b0-dbeb-429e-a904-866f615de2c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:55.847536 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ec39b0-dbeb-429e-a904-866f615de2c6 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:55.852020 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e85a881-5348-4477-bd5a-a28f9054e75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:55.853128 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85a881-5348-4477-bd5a-a28f9054e75f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:55.853611 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85a881-5348-4477-bd5a-a28f9054e75f admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:55.854093 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e85a881-5348-4477-bd5a-a28f9054e75f admin admin] PATH_INFO: `/v3/projects/8901313d60ea42378d4e1c2d8dfe40a9` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:55.855446 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e85a881-5348-4477-bd5a-a28f9054e75f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8901313d60ea42378d4e1c2d8dfe40a9)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:55.858136 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e85a881-5348-4477-bd5a-a28f9054e75f admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:55.862147 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 984/1933] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:55.863136 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ec39b0-dbeb-429e-a904-866f615de2c6 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:55.873208 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ec39b0-dbeb-429e-a904-866f615de2c6 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:55.878933 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46145d50-7d70-431c-8d9c-24c1b1049281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:55.879890 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46145d50-7d70-431c-8d9c-24c1b1049281 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:55.880276 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46145d50-7d70-431c-8d9c-24c1b1049281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:55.880445 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46145d50-7d70-431c-8d9c-24c1b1049281 admin admin] PATH_INFO: `/v3/projects/3768d22800fa412384957b44b4fa6ab3/users/6a30d684ce9b441b8fe1fa4d5aba9a39/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:55.896870 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46145d50-7d70-431c-8d9c-24c1b1049281 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3768d22800fa412384957b44b4fa6ab3, user_id=6a30d684ce9b441b8fe1fa4d5aba9a39, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:55.899672 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46145d50-7d70-431c-8d9c-24c1b1049281 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:55.984201 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 985/1934] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:55 2022] PUT /identity/v3/projects/3768d22800fa412384957b44b4fa6ab3/users/6a30d684ce9b441b8fe1fa4d5aba9a39/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:13:55.988137 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46145d50-7d70-431c-8d9c-24c1b1049281 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:55.992848 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-2e85a881-5348-4477-bd5a-a28f9054e75f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8901313d60ea42378d4e1c2d8dfe40a9', 'request_id': 'req-2e85a881-5348-4477-bd5a-a28f9054e75f'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:56.002958 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-2e85a881-5348-4477-bd5a-a28f9054e75f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8901313d60ea42378d4e1c2d8dfe40a9', 'request_id': 'req-2e85a881-5348-4477-bd5a-a28f9054e75f'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:56.061764 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46145d50-7d70-431c-8d9c-24c1b1049281 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:56.111430 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-2e85a881-5348-4477-bd5a-a28f9054e75f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8901313d60ea42378d4e1c2d8dfe40a9', 'request_id': 'req-2e85a881-5348-4477-bd5a-a28f9054e75f'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:56.119496 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bc042b-a968-4926-8b1d-597291f2c554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:56.120230 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc042b-a968-4926-8b1d-597291f2c554 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:56.121406 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc042b-a968-4926-8b1d-597291f2c554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:56.121816 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bc042b-a968-4926-8b1d-597291f2c554 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:56.122776 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bc042b-a968-4926-8b1d-597291f2c554 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:56.124988 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-2e85a881-5348-4477-bd5a-a28f9054e75f admin admin] The token cache is being invalidate because project 8901313d60ea42378d4e1c2d8dfe40a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:13:56.125327 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bc042b-a968-4926-8b1d-597291f2c554 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:56.125747 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-2e85a881-5348-4477-bd5a-a28f9054e75f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:56.131946 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-2e85a881-5348-4477-bd5a-a28f9054e75f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:56.131946 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 950/1935] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:13:55 2022] DELETE /identity/v3/projects/8901313d60ea42378d4e1c2d8dfe40a9 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:13:56.133263 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e85a881-5348-4477-bd5a-a28f9054e75f admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:56.134932 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-2e85a881-5348-4477-bd5a-a28f9054e75f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:56.140441 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 986/1936] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:55 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:13:56.144478 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bc042b-a968-4926-8b1d-597291f2c554 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:56.148633 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e85a881-5348-4477-bd5a-a28f9054e75f admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:56.149290 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f9bc042b-a968-4926-8b1d-597291f2c554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:56.153759 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce16cbf-23b4-41e8-965f-475a23b7ab93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:56.155147 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce16cbf-23b4-41e8-965f-475a23b7ab93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:56.155432 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce16cbf-23b4-41e8-965f-475a23b7ab93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:56.155702 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce16cbf-23b4-41e8-965f-475a23b7ab93 admin admin] PATH_INFO: `/v3/users/71a93e1f76dc427d906c3ed4bd8249ce` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:56.156938 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce16cbf-23b4-41e8-965f-475a23b7ab93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71a93e1f76dc427d906c3ed4bd8249ce)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:56.158844 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce16cbf-23b4-41e8-965f-475a23b7ab93 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:56.162705 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bc042b-a968-4926-8b1d-597291f2c554 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:56.167498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea8d0e7-0570-4e12-9141-3b09de3f09f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:56.168456 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea8d0e7-0570-4e12-9141-3b09de3f09f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:56.168742 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea8d0e7-0570-4e12-9141-3b09de3f09f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:56.169021 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea8d0e7-0570-4e12-9141-3b09de3f09f5 admin admin] PATH_INFO: `/v3/projects/3768d22800fa412384957b44b4fa6ab3/users/6a30d684ce9b441b8fe1fa4d5aba9a39/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:56.171760 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea8d0e7-0570-4e12-9141-3b09de3f09f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3768d22800fa412384957b44b4fa6ab3, user_id=6a30d684ce9b441b8fe1fa4d5aba9a39, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:56.174448 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea8d0e7-0570-4e12-9141-3b09de3f09f5 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:56.185022 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 987/1937] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:56 2022] PUT /identity/v3/projects/3768d22800fa412384957b44b4fa6ab3/users/6a30d684ce9b441b8fe1fa4d5aba9a39/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:13:56.195393 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87972f1-74df-4bb7-b8e1-6e5b64020292 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:56.195936 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87972f1-74df-4bb7-b8e1-6e5b64020292 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:56.196231 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87972f1-74df-4bb7-b8e1-6e5b64020292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:56.223345 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-e87972f1-74df-4bb7-b8e1-6e5b64020292 None None] MFA Rules not processed for user `6a30d684ce9b441b8fe1fa4d5aba9a39`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:56.224916 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-e87972f1-74df-4bb7-b8e1-6e5b64020292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:56.294636 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 988/1938] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:56 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:13:56.306053 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87972f1-74df-4bb7-b8e1-6e5b64020292 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:56.309220 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-e87972f1-74df-4bb7-b8e1-6e5b64020292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:56.370151 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87972f1-74df-4bb7-b8e1-6e5b64020292 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:56.373999 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10041655-8450-46cd-93ba-7b6cfcdac547 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:56.374660 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10041655-8450-46cd-93ba-7b6cfcdac547 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:56.374949 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10041655-8450-46cd-93ba-7b6cfcdac547 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:56.375229 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10041655-8450-46cd-93ba-7b6cfcdac547 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:56.385110 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10041655-8450-46cd-93ba-7b6cfcdac547 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:56.386550 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10041655-8450-46cd-93ba-7b6cfcdac547 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:56.389891 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6ce16cbf-23b4-41e8-965f-475a23b7ab93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71a93e1f76dc427d906c3ed4bd8249ce'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:56.397597 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 989/1939] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:56 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 17:13:56.413578 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6ce16cbf-23b4-41e8-965f-475a23b7ab93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71a93e1f76dc427d906c3ed4bd8249ce'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:56.421819 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6ce16cbf-23b4-41e8-965f-475a23b7ab93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71a93e1f76dc427d906c3ed4bd8249ce'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:56.422872 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6ce16cbf-23b4-41e8-965f-475a23b7ab93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71a93e1f76dc427d906c3ed4bd8249ce'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:56.431607 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:56.443524 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10041655-8450-46cd-93ba-7b6cfcdac547 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:56.445082 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-10041655-8450-46cd-93ba-7b6cfcdac547 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:56.457828 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10041655-8450-46cd-93ba-7b6cfcdac547 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:56.462114 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69124c-016c-429c-94b7-bd83458c46eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:56.462847 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69124c-016c-429c-94b7-bd83458c46eb service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:56.463753 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69124c-016c-429c-94b7-bd83458c46eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:56.463867 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e69124c-016c-429c-94b7-bd83458c46eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:56.465849 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-8e69124c-016c-429c-94b7-bd83458c46eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:56.505501 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6ce16cbf-23b4-41e8-965f-475a23b7ab93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71a93e1f76dc427d906c3ed4bd8249ce'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:56.512432 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 951/1940] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:13:56 2022] DELETE /identity/v3/users/71a93e1f76dc427d906c3ed4bd8249ce => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:56.548064 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e69124c-016c-429c-94b7-bd83458c46eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:56.549820 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e69124c-016c-429c-94b7-bd83458c46eb service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:56.551553 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce16cbf-23b4-41e8-965f-475a23b7ab93 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:56.553225 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-6ce16cbf-23b4-41e8-965f-475a23b7ab93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:56.561756 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 990/1941] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:56 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 17:13:56.668184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce16cbf-23b4-41e8-965f-475a23b7ab93 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:56.674129 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7143a694-0098-4473-8efd-24dc52d865e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:56.674129 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143a694-0098-4473-8efd-24dc52d865e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:56.674465 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143a694-0098-4473-8efd-24dc52d865e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:56.674780 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7143a694-0098-4473-8efd-24dc52d865e5 admin admin] PATH_INFO: `/v3/users/c2b09349284d4c8f99876ae7277095b9` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:56.676118 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7143a694-0098-4473-8efd-24dc52d865e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2b09349284d4c8f99876ae7277095b9)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:56.678059 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7143a694-0098-4473-8efd-24dc52d865e5 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:56.780084 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:56.811868 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-7143a694-0098-4473-8efd-24dc52d865e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2b09349284d4c8f99876ae7277095b9'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:56.824133 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-7143a694-0098-4473-8efd-24dc52d865e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2b09349284d4c8f99876ae7277095b9'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:56.832205 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-7143a694-0098-4473-8efd-24dc52d865e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2b09349284d4c8f99876ae7277095b9'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:56.832966 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-7143a694-0098-4473-8efd-24dc52d865e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2b09349284d4c8f99876ae7277095b9'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:56.914182 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69124c-016c-429c-94b7-bd83458c46eb service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:56.923379 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e69124c-016c-429c-94b7-bd83458c46eb service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:56.937360 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-7143a694-0098-4473-8efd-24dc52d865e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2b09349284d4c8f99876ae7277095b9'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:56.948347 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 952/1942] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:13:56 2022] DELETE /identity/v3/users/c2b09349284d4c8f99876ae7277095b9 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:13:56.954166 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79a86f-5284-4d37-950e-f85fb46ad3ac None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:56.955133 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79a86f-5284-4d37-950e-f85fb46ad3ac None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:56.956989 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de79a86f-5284-4d37-950e-f85fb46ad3ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:56.979658 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-de79a86f-5284-4d37-950e-f85fb46ad3ac None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:56.981801 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-de79a86f-5284-4d37-950e-f85fb46ad3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:56.988348 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6782f8a-9078-43c3-98eb-d23cc56283b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:56.989133 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6782f8a-9078-43c3-98eb-d23cc56283b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:56.989445 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6782f8a-9078-43c3-98eb-d23cc56283b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:56.989719 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6782f8a-9078-43c3-98eb-d23cc56283b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:56.993506 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d6782f8a-9078-43c3-98eb-d23cc56283b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:57.054103 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 953/1943] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:13:57.055095 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de79a86f-5284-4d37-950e-f85fb46ad3ac None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:57.056436 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-de79a86f-5284-4d37-950e-f85fb46ad3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:57.071032 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de79a86f-5284-4d37-950e-f85fb46ad3ac None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:57.071503 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6782f8a-9078-43c3-98eb-d23cc56283b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:57.073592 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6782f8a-9078-43c3-98eb-d23cc56283b9 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:57.076658 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292ccaac-0b24-4a6e-a723-b8d26e97151c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:57.077708 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292ccaac-0b24-4a6e-a723-b8d26e97151c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:57.078309 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292ccaac-0b24-4a6e-a723-b8d26e97151c admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:57.078739 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292ccaac-0b24-4a6e-a723-b8d26e97151c admin admin] PATH_INFO: `/v3/projects/c2ba1de3f391450aa4e744068d800fbd` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:57.081031 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292ccaac-0b24-4a6e-a723-b8d26e97151c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2ba1de3f391450aa4e744068d800fbd)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:57.082935 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292ccaac-0b24-4a6e-a723-b8d26e97151c admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:57.091925 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 991/1944] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:56 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 17:13:57.093181 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6782f8a-9078-43c3-98eb-d23cc56283b9 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:57.094752 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d6782f8a-9078-43c3-98eb-d23cc56283b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:57.166306 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6782f8a-9078-43c3-98eb-d23cc56283b9 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:57.235627 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff31f3a-bbfb-4159-9e64-9b2cb79ece9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:57.236338 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff31f3a-bbfb-4159-9e64-9b2cb79ece9a service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:57.236624 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff31f3a-bbfb-4159-9e64-9b2cb79ece9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:57.236903 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff31f3a-bbfb-4159-9e64-9b2cb79ece9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:57.266764 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff31f3a-bbfb-4159-9e64-9b2cb79ece9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:57.268243 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff31f3a-bbfb-4159-9e64-9b2cb79ece9a service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:57.271378 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-292ccaac-0b24-4a6e-a723-b8d26e97151c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2ba1de3f391450aa4e744068d800fbd', 'request_id': 'req-292ccaac-0b24-4a6e-a723-b8d26e97151c'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:57.288890 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-292ccaac-0b24-4a6e-a723-b8d26e97151c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2ba1de3f391450aa4e744068d800fbd', 'request_id': 'req-292ccaac-0b24-4a6e-a723-b8d26e97151c'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:57.300071 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 992/1945] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:13:57.304010 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d381551-324d-4096-ad89-513997051cb2 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:57.304437 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d381551-324d-4096-ad89-513997051cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:57.304729 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d381551-324d-4096-ad89-513997051cb2 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:57.307538 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 993/1946] 10.222.0.40 () {54 vars in 921 bytes} [Wed Jul 6 17:13:57 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:13:57.319815 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8736236-b8dd-4d14-adad-836dee31949d None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:57.320111 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8736236-b8dd-4d14-adad-836dee31949d None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:57.320444 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8736236-b8dd-4d14-adad-836dee31949d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:57.342977 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-c8736236-b8dd-4d14-adad-836dee31949d None None] MFA Rules not processed for user `af26ef96dc9d47dc91549b736939268e`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:57.344250 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c8736236-b8dd-4d14-adad-836dee31949d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:57.420974 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 994/1947] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Jul 6 17:13:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:13:57.440197 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8736236-b8dd-4d14-adad-836dee31949d None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:57.443682 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c8736236-b8dd-4d14-adad-836dee31949d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:57.458250 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8736236-b8dd-4d14-adad-836dee31949d None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:57.467550 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8629be-a302-4d72-bb3b-585eb3a67d7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:57.468103 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8629be-a302-4d72-bb3b-585eb3a67d7d service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:57.468451 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8629be-a302-4d72-bb3b-585eb3a67d7d service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:57.468858 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8629be-a302-4d72-bb3b-585eb3a67d7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:57.479147 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8629be-a302-4d72-bb3b-585eb3a67d7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:57.480802 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8629be-a302-4d72-bb3b-585eb3a67d7d service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:57.493142 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 995/1948] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:13:57.522284 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-292ccaac-0b24-4a6e-a723-b8d26e97151c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2ba1de3f391450aa4e744068d800fbd', 'request_id': 'req-292ccaac-0b24-4a6e-a723-b8d26e97151c'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:57.540493 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-292ccaac-0b24-4a6e-a723-b8d26e97151c admin admin] The token cache is being invalidate because project c2ba1de3f391450aa4e744068d800fbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:13:57.540914 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-292ccaac-0b24-4a6e-a723-b8d26e97151c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:57.541836 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-292ccaac-0b24-4a6e-a723-b8d26e97151c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:57.548954 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b902f0-e8a3-4a33-96cb-80b6ec02f13a None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:57.549461 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b902f0-e8a3-4a33-96cb-80b6ec02f13a None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:57.549822 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b902f0-e8a3-4a33-96cb-80b6ec02f13a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:57.551676 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 954/1949] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:13:57 2022] DELETE /identity/v3/projects/c2ba1de3f391450aa4e744068d800fbd => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:57.576016 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-72b902f0-e8a3-4a33-96cb-80b6ec02f13a None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:57.580775 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-72b902f0-e8a3-4a33-96cb-80b6ec02f13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:57.645096 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:57.648782 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 996/1950] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:57.658339 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec8cfa-30eb-452e-8846-8f743bbbce70 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:57.659029 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec8cfa-30eb-452e-8846-8f743bbbce70 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:57.659529 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec8cfa-30eb-452e-8846-8f743bbbce70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:57.694365 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-22ec8cfa-30eb-452e-8846-8f743bbbce70 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:57.695665 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-22ec8cfa-30eb-452e-8846-8f743bbbce70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:57.733784 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 955/1951] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:57.739841 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b902f0-e8a3-4a33-96cb-80b6ec02f13a None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:57.752533 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b902f0-e8a3-4a33-96cb-80b6ec02f13a None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:57.762479 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1dcebd-8a0c-46e9-af36-d3ad8732a8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:57.763276 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1dcebd-8a0c-46e9-af36-d3ad8732a8cd admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:57.763451 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1dcebd-8a0c-46e9-af36-d3ad8732a8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:57.764396 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1dcebd-8a0c-46e9-af36-d3ad8732a8cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:57.765148 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1dcebd-8a0c-46e9-af36-d3ad8732a8cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:13:57.766402 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1dcebd-8a0c-46e9-af36-d3ad8732a8cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:57.783200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1dcebd-8a0c-46e9-af36-d3ad8732a8cd admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:57.808841 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 997/1952] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:13:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 17:13:57.809641 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec8cfa-30eb-452e-8846-8f743bbbce70 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:57.821227 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec8cfa-30eb-452e-8846-8f743bbbce70 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:57.826092 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b24393-fa3c-40a4-865c-99ae886b2ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:57.826985 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b24393-fa3c-40a4-865c-99ae886b2ca8 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:57.827466 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b24393-fa3c-40a4-865c-99ae886b2ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:57.827781 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b24393-fa3c-40a4-865c-99ae886b2ca8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:57.828486 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b24393-fa3c-40a4-865c-99ae886b2ca8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:57.830677 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b24393-fa3c-40a4-865c-99ae886b2ca8 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:57.846648 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1dcebd-8a0c-46e9-af36-d3ad8732a8cd admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:57.848604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-6e1dcebd-8a0c-46e9-af36-d3ad8732a8cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:57.865595 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1dcebd-8a0c-46e9-af36-d3ad8732a8cd admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:57.877455 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff8569-ab69-447e-a457-03d93084d774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:57.878723 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff8569-ab69-447e-a457-03d93084d774 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:57.878723 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff8569-ab69-447e-a457-03d93084d774 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:57.878947 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ff8569-ab69-447e-a457-03d93084d774 admin admin] PATH_INFO: `/v3/projects/3c400aae4350471291c678e2e4972aff` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:57.880551 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ff8569-ab69-447e-a457-03d93084d774 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c400aae4350471291c678e2e4972aff)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:57.882891 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ff8569-ab69-447e-a457-03d93084d774 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:57.925208 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 956/1953] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:13:57 2022] POST /identity/v3/projects => generated 359 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:57.927665 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b24393-fa3c-40a4-865c-99ae886b2ca8 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:58.025911 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:58.049626 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-02ff8569-ab69-447e-a457-03d93084d774 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c400aae4350471291c678e2e4972aff', 'request_id': 'req-02ff8569-ab69-447e-a457-03d93084d774'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:58.061729 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-02ff8569-ab69-447e-a457-03d93084d774 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c400aae4350471291c678e2e4972aff', 'request_id': 'req-02ff8569-ab69-447e-a457-03d93084d774'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:58.068378 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b24393-fa3c-40a4-865c-99ae886b2ca8 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:58.069765 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-02ff8569-ab69-447e-a457-03d93084d774 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c400aae4350471291c678e2e4972aff', 'request_id': 'req-02ff8569-ab69-447e-a457-03d93084d774'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:58.137386 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3183c964-5b1b-4090-b706-cff10e457857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:58.138074 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183c964-5b1b-4090-b706-cff10e457857 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:58.138356 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183c964-5b1b-4090-b706-cff10e457857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:58.138656 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3183c964-5b1b-4090-b706-cff10e457857 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:58.139380 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3183c964-5b1b-4090-b706-cff10e457857 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:58.141375 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3183c964-5b1b-4090-b706-cff10e457857 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:58.202973 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-02ff8569-ab69-447e-a457-03d93084d774 admin admin] The token cache is being invalidate because project 3c400aae4350471291c678e2e4972aff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:13:58.203262 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-02ff8569-ab69-447e-a457-03d93084d774 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:58.204024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-02ff8569-ab69-447e-a457-03d93084d774 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:58.206293 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 998/1954] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:13:57 2022] DELETE /identity/v3/projects/3c400aae4350471291c678e2e4972aff => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:13:58.210343 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff8569-ab69-447e-a457-03d93084d774 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:58.211800 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-02ff8569-ab69-447e-a457-03d93084d774 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:58.233134 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 957/1955] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:13:57 2022] POST /identity/v3/users => generated 280 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:13:58.234044 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3183c964-5b1b-4090-b706-cff10e457857 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:58.237313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-3183c964-5b1b-4090-b706-cff10e457857 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:58.242592 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ff8569-ab69-447e-a457-03d93084d774 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:58.249230 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a4c33-80df-416a-a36c-0e4c6d398721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:58.250673 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a4c33-80df-416a-a36c-0e4c6d398721 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:58.251253 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a4c33-80df-416a-a36c-0e4c6d398721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:58.251757 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a4c33-80df-416a-a36c-0e4c6d398721 admin admin] PATH_INFO: `/v3/users/6a30d684ce9b441b8fe1fa4d5aba9a39` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:58.258725 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a4c33-80df-416a-a36c-0e4c6d398721 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a30d684ce9b441b8fe1fa4d5aba9a39)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:58.258864 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3183c964-5b1b-4090-b706-cff10e457857 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:58.258968 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a4c33-80df-416a-a36c-0e4c6d398721 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:58.261880 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035cfd57-6151-4da0-a28b-2fae75824803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:58.263536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035cfd57-6151-4da0-a28b-2fae75824803 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:58.264715 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035cfd57-6151-4da0-a28b-2fae75824803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:58.265070 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035cfd57-6151-4da0-a28b-2fae75824803 admin admin] PATH_INFO: `/v3/users/c34dfe8ae9fc4c268bfb2f627a43b046` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:58.266934 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035cfd57-6151-4da0-a28b-2fae75824803 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c34dfe8ae9fc4c268bfb2f627a43b046)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:58.269434 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035cfd57-6151-4da0-a28b-2fae75824803 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:58.421847 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:13:58.540795 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-ba5a4c33-80df-416a-a36c-0e4c6d398721 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a30d684ce9b441b8fe1fa4d5aba9a39'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:58.541307 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-035cfd57-6151-4da0-a28b-2fae75824803 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c34dfe8ae9fc4c268bfb2f627a43b046'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:58.548950 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-ba5a4c33-80df-416a-a36c-0e4c6d398721 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a30d684ce9b441b8fe1fa4d5aba9a39'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:58.549710 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-ba5a4c33-80df-416a-a36c-0e4c6d398721 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a30d684ce9b441b8fe1fa4d5aba9a39'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:58.554231 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-035cfd57-6151-4da0-a28b-2fae75824803 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c34dfe8ae9fc4c268bfb2f627a43b046'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:58.563098 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-035cfd57-6151-4da0-a28b-2fae75824803 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c34dfe8ae9fc4c268bfb2f627a43b046'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:58.563237 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-ba5a4c33-80df-416a-a36c-0e4c6d398721 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a30d684ce9b441b8fe1fa4d5aba9a39'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:58.563851 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-035cfd57-6151-4da0-a28b-2fae75824803 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c34dfe8ae9fc4c268bfb2f627a43b046'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:58.630990 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-ba5a4c33-80df-416a-a36c-0e4c6d398721 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a30d684ce9b441b8fe1fa4d5aba9a39'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:58.637529 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 999/1956] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:13:58 2022] DELETE /identity/v3/users/6a30d684ce9b441b8fe1fa4d5aba9a39 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:58.638409 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a4c33-80df-416a-a36c-0e4c6d398721 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:58.640044 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-ba5a4c33-80df-416a-a36c-0e4c6d398721 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:58.710931 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-035cfd57-6151-4da0-a28b-2fae75824803 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c34dfe8ae9fc4c268bfb2f627a43b046'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:58.723134 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a4c33-80df-416a-a36c-0e4c6d398721 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:58.724543 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 958/1957] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:13:58 2022] DELETE /identity/v3/users/c34dfe8ae9fc4c268bfb2f627a43b046 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:13:58.734766 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07c128-d8d2-4793-815d-16aadff0c3ad None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:58.734766 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07c128-d8d2-4793-815d-16aadff0c3ad None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:58.734766 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07c128-d8d2-4793-815d-16aadff0c3ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:58.750940 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-8e07c128-d8d2-4793-815d-16aadff0c3ad None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:58.752305 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-8e07c128-d8d2-4793-815d-16aadff0c3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:58.788933 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc458fa-48a0-416a-96b3-9094503da0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:58.789679 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc458fa-48a0-416a-96b3-9094503da0f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:58.789965 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc458fa-48a0-416a-96b3-9094503da0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:58.790249 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc458fa-48a0-416a-96b3-9094503da0f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:58.791047 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc458fa-48a0-416a-96b3-9094503da0f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:58.792679 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc458fa-48a0-416a-96b3-9094503da0f6 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:58.807704 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1000/1958] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:58.808489 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc458fa-48a0-416a-96b3-9094503da0f6 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:58.810187 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-efc458fa-48a0-416a-96b3-9094503da0f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:58.822901 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 959/1959] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:13:58.823494 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc458fa-48a0-416a-96b3-9094503da0f6 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:58.832561 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e07c128-d8d2-4793-815d-16aadff0c3ad None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:58.833954 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-8e07c128-d8d2-4793-815d-16aadff0c3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:58.834770 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b225c-7393-4cca-98df-ad12ebdf7911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:58.835581 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b225c-7393-4cca-98df-ad12ebdf7911 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:58.835973 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b225c-7393-4cca-98df-ad12ebdf7911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:58.836325 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b225c-7393-4cca-98df-ad12ebdf7911 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:58.837138 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9b225c-7393-4cca-98df-ad12ebdf7911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:58.838810 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9b225c-7393-4cca-98df-ad12ebdf7911 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:58.853890 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1001/1960] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:13:58.854856 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b225c-7393-4cca-98df-ad12ebdf7911 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:58.866749 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b225c-7393-4cca-98df-ad12ebdf7911 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:58.884715 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c606b921-e4d3-4edd-b204-0b7d9f01cccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:58.885631 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c606b921-e4d3-4edd-b204-0b7d9f01cccd admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:58.885917 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c606b921-e4d3-4edd-b204-0b7d9f01cccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:58.886227 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c606b921-e4d3-4edd-b204-0b7d9f01cccd admin admin] PATH_INFO: `/v3/projects/550bd9c90de74021a41c06779ec2d136/users/8311b864efec4f9ea7c34fad55ffe157/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:58.909634 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c606b921-e4d3-4edd-b204-0b7d9f01cccd admin admin] RBAC: Authorizing `identity:create_grant(project_id=550bd9c90de74021a41c06779ec2d136, user_id=8311b864efec4f9ea7c34fad55ffe157, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:58.912563 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c606b921-e4d3-4edd-b204-0b7d9f01cccd admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:58.916332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e07c128-d8d2-4793-815d-16aadff0c3ad None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:58.920261 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff30cce3-47a3-47c2-ab6c-927050508ee9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:58.921297 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30cce3-47a3-47c2-ab6c-927050508ee9 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:58.921591 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30cce3-47a3-47c2-ab6c-927050508ee9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:58.921876 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff30cce3-47a3-47c2-ab6c-927050508ee9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:58.931764 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff30cce3-47a3-47c2-ab6c-927050508ee9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:58.933209 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff30cce3-47a3-47c2-ab6c-927050508ee9 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:58.944258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 960/1961] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 17:13:58.969868 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1002/1962] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:13:58 2022] PUT /identity/v3/projects/550bd9c90de74021a41c06779ec2d136/users/8311b864efec4f9ea7c34fad55ffe157/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:13:58.980386 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66618ad-1536-4344-9729-6739dfeab988 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:58.980814 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66618ad-1536-4344-9729-6739dfeab988 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:58.981103 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66618ad-1536-4344-9729-6739dfeab988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:59.009429 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-c66618ad-1536-4344-9729-6739dfeab988 None None] MFA Rules not processed for user `8311b864efec4f9ea7c34fad55ffe157`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:59.010661 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c66618ad-1536-4344-9729-6739dfeab988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:59.078603 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 961/1963] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:58 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:13:59.111781 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c606b921-e4d3-4edd-b204-0b7d9f01cccd admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:59.113450 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c606b921-e4d3-4edd-b204-0b7d9f01cccd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:59.145292 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66618ad-1536-4344-9729-6739dfeab988 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:59.149175 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c66618ad-1536-4344-9729-6739dfeab988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:59.209470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c606b921-e4d3-4edd-b204-0b7d9f01cccd admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:59.219525 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cedfc81-6a3f-4a4a-9336-557577a29ef8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:59.219866 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cedfc81-6a3f-4a4a-9336-557577a29ef8 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:59.220066 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cedfc81-6a3f-4a4a-9336-557577a29ef8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:59.220346 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cedfc81-6a3f-4a4a-9336-557577a29ef8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:59.241537 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cedfc81-6a3f-4a4a-9336-557577a29ef8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:59.243020 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cedfc81-6a3f-4a4a-9336-557577a29ef8 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:59.268427 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1003/1964] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:59 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:13:59.276748 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cedfc81-6a3f-4a4a-9336-557577a29ef8 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:59.276925 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66618ad-1536-4344-9729-6739dfeab988 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:59.281233 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97456fa0-4ba4-402b-a4a6-ba05aa019c56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:59.282186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97456fa0-4ba4-402b-a4a6-ba05aa019c56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:59.282543 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97456fa0-4ba4-402b-a4a6-ba05aa019c56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:59.282892 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97456fa0-4ba4-402b-a4a6-ba05aa019c56 admin admin] PATH_INFO: `/v3/projects/3768d22800fa412384957b44b4fa6ab3` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:59.284260 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97456fa0-4ba4-402b-a4a6-ba05aa019c56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3768d22800fa412384957b44b4fa6ab3)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:59.286244 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97456fa0-4ba4-402b-a4a6-ba05aa019c56 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:59.295391 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cedfc81-6a3f-4a4a-9336-557577a29ef8 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:59.302737 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd0bc19-78c2-4dd1-bd5e-29d01406aaa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:59.303827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0bc19-78c2-4dd1-bd5e-29d01406aaa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:59.304497 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0bc19-78c2-4dd1-bd5e-29d01406aaa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:59.304854 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0bc19-78c2-4dd1-bd5e-29d01406aaa7 admin admin] PATH_INFO: `/v3/users/8311b864efec4f9ea7c34fad55ffe157` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:59.306499 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd0bc19-78c2-4dd1-bd5e-29d01406aaa7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8311b864efec4f9ea7c34fad55ffe157)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:59.314571 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd0bc19-78c2-4dd1-bd5e-29d01406aaa7 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:59.431058 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-97456fa0-4ba4-402b-a4a6-ba05aa019c56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3768d22800fa412384957b44b4fa6ab3', 'request_id': 'req-97456fa0-4ba4-402b-a4a6-ba05aa019c56'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:59.441082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-97456fa0-4ba4-402b-a4a6-ba05aa019c56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3768d22800fa412384957b44b4fa6ab3', 'request_id': 'req-97456fa0-4ba4-402b-a4a6-ba05aa019c56'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:59.543600 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-97456fa0-4ba4-402b-a4a6-ba05aa019c56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3768d22800fa412384957b44b4fa6ab3', 'request_id': 'req-97456fa0-4ba4-402b-a4a6-ba05aa019c56'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:59.559124 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-dfd0bc19-78c2-4dd1-bd5e-29d01406aaa7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8311b864efec4f9ea7c34fad55ffe157'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:59.562876 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-97456fa0-4ba4-402b-a4a6-ba05aa019c56 admin admin] The token cache is being invalidate because project 3768d22800fa412384957b44b4fa6ab3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:13:59.563169 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-97456fa0-4ba4-402b-a4a6-ba05aa019c56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:59.566335 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-97456fa0-4ba4-402b-a4a6-ba05aa019c56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:59.569259 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-dfd0bc19-78c2-4dd1-bd5e-29d01406aaa7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8311b864efec4f9ea7c34fad55ffe157'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:59.570197 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-dfd0bc19-78c2-4dd1-bd5e-29d01406aaa7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8311b864efec4f9ea7c34fad55ffe157'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:59.577138 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 962/1965] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:13:59 2022] DELETE /identity/v3/projects/3768d22800fa412384957b44b4fa6ab3 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:13:59.578824 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128728c-aef6-4f17-8f74-803fdf380eff None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:59.579121 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128728c-aef6-4f17-8f74-803fdf380eff None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:59.579401 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128728c-aef6-4f17-8f74-803fdf380eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:59.584781 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-dfd0bc19-78c2-4dd1-bd5e-29d01406aaa7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8311b864efec4f9ea7c34fad55ffe157'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:59.609308 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-e128728c-aef6-4f17-8f74-803fdf380eff None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:59.610609 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e128728c-aef6-4f17-8f74-803fdf380eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:59.677320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-dfd0bc19-78c2-4dd1-bd5e-29d01406aaa7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8311b864efec4f9ea7c34fad55ffe157'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:59.679608 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 963/1966] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:59.686208 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1004/1967] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:13:59 2022] DELETE /identity/v3/users/8311b864efec4f9ea7c34fad55ffe157 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:13:59.687332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e128728c-aef6-4f17-8f74-803fdf380eff None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:59.692525 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87df461-425a-4966-9954-499f18befe16 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:59.692525 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87df461-425a-4966-9954-499f18befe16 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:59.692525 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87df461-425a-4966-9954-499f18befe16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:59.692748 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e128728c-aef6-4f17-8f74-803fdf380eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:59.700834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e128728c-aef6-4f17-8f74-803fdf380eff None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:59.709903 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-f87df461-425a-4966-9954-499f18befe16 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:59.711706 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f87df461-425a-4966-9954-499f18befe16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:59.782781 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b676986-1c96-4d8d-9b33-c3c6c39ee489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:59.783651 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b676986-1c96-4d8d-9b33-c3c6c39ee489 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:59.784008 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b676986-1c96-4d8d-9b33-c3c6c39ee489 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:59.784289 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b676986-1c96-4d8d-9b33-c3c6c39ee489 admin admin] PATH_INFO: `/v3/projects/16cdde3692144dc08a3d3915d903db02` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:59.785534 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b676986-1c96-4d8d-9b33-c3c6c39ee489 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16cdde3692144dc08a3d3915d903db02)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:59.787444 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b676986-1c96-4d8d-9b33-c3c6c39ee489 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:59.789768 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1005/1968] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:59.790989 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc49088-a3f7-4d7a-872d-81dc715d2bde None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:59.791284 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc49088-a3f7-4d7a-872d-81dc715d2bde None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:59.791570 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc49088-a3f7-4d7a-872d-81dc715d2bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:59.810574 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-fcc49088-a3f7-4d7a-872d-81dc715d2bde None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:13:59.811903 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-fcc49088-a3f7-4d7a-872d-81dc715d2bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:59.819585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1006/1969] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:13:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:13:59.820378 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc49088-a3f7-4d7a-872d-81dc715d2bde None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:59.829631 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc49088-a3f7-4d7a-872d-81dc715d2bde None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:59.834261 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28981e4d-5479-405b-8883-2b5055f6b008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:59.834905 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28981e4d-5479-405b-8883-2b5055f6b008 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:59.835190 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28981e4d-5479-405b-8883-2b5055f6b008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:59.835460 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28981e4d-5479-405b-8883-2b5055f6b008 admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:13:59.836210 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28981e4d-5479-405b-8883-2b5055f6b008 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:13:59.836513 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28981e4d-5479-405b-8883-2b5055f6b008 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:13:59.838089 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28981e4d-5479-405b-8883-2b5055f6b008 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:13:59.849923 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1007/1970] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:13:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:13:59.850680 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28981e4d-5479-405b-8883-2b5055f6b008 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:13:59.852242 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-28981e4d-5479-405b-8883-2b5055f6b008 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:13:59.899504 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-9b676986-1c96-4d8d-9b33-c3c6c39ee489 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16cdde3692144dc08a3d3915d903db02', 'request_id': 'req-9b676986-1c96-4d8d-9b33-c3c6c39ee489'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:59.906389 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-9b676986-1c96-4d8d-9b33-c3c6c39ee489 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16cdde3692144dc08a3d3915d903db02', 'request_id': 'req-9b676986-1c96-4d8d-9b33-c3c6c39ee489'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:13:59.907017 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28981e4d-5479-405b-8883-2b5055f6b008 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:13:59.953667 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b96c167-3d59-4e42-8f78-abb27bc5dce5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:13:59.954315 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96c167-3d59-4e42-8f78-abb27bc5dce5 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:13:59.954636 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96c167-3d59-4e42-8f78-abb27bc5dce5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:13:59.954912 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96c167-3d59-4e42-8f78-abb27bc5dce5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:00.013183 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b96c167-3d59-4e42-8f78-abb27bc5dce5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:00.015129 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b96c167-3d59-4e42-8f78-abb27bc5dce5 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:00.019585 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-9b676986-1c96-4d8d-9b33-c3c6c39ee489 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16cdde3692144dc08a3d3915d903db02', 'request_id': 'req-9b676986-1c96-4d8d-9b33-c3c6c39ee489'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:00.024415 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1008/1971] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:13:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:14:00.025205 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b96c167-3d59-4e42-8f78-abb27bc5dce5 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:00.032299 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-9b676986-1c96-4d8d-9b33-c3c6c39ee489 admin admin] The token cache is being invalidate because project 16cdde3692144dc08a3d3915d903db02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:14:00.032586 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-9b676986-1c96-4d8d-9b33-c3c6c39ee489 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:00.033644 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-9b676986-1c96-4d8d-9b33-c3c6c39ee489 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:00.036304 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b96c167-3d59-4e42-8f78-abb27bc5dce5 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:00.041067 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 964/1972] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:13:59 2022] DELETE /identity/v3/projects/16cdde3692144dc08a3d3915d903db02 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:00.041605 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b676986-1c96-4d8d-9b33-c3c6c39ee489 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:00.043839 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-9b676986-1c96-4d8d-9b33-c3c6c39ee489 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:00.044342 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c1056d-f45e-4df9-94d0-4f18a9667a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:00.046022 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1056d-f45e-4df9-94d0-4f18a9667a51 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:00.046022 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1056d-f45e-4df9-94d0-4f18a9667a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:00.046022 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c1056d-f45e-4df9-94d0-4f18a9667a51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:00.046393 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c1056d-f45e-4df9-94d0-4f18a9667a51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:00.048318 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c1056d-f45e-4df9-94d0-4f18a9667a51 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:00.059848 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b676986-1c96-4d8d-9b33-c3c6c39ee489 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:00.066319 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27df099-466c-48f8-b752-07378b0232c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:00.067306 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27df099-466c-48f8-b752-07378b0232c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:00.067678 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27df099-466c-48f8-b752-07378b0232c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:00.068173 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27df099-466c-48f8-b752-07378b0232c1 admin admin] PATH_INFO: `/v3/users/915a1b3393bf4384a2d4d9a2bb8559e8` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:00.069692 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27df099-466c-48f8-b752-07378b0232c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=915a1b3393bf4384a2d4d9a2bb8559e8)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:00.071613 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27df099-466c-48f8-b752-07378b0232c1 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:00.147278 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1009/1973] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:14:00 2022] POST /identity/v3/projects => generated 409 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:14:00.152002 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c1056d-f45e-4df9-94d0-4f18a9667a51 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:00.153844 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-27c1056d-f45e-4df9-94d0-4f18a9667a51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:00.169623 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:00.219679 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c1056d-f45e-4df9-94d0-4f18a9667a51 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:00.224279 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db2aa3-98a4-49b2-8e3f-fadafc2c677c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:00.224956 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db2aa3-98a4-49b2-8e3f-fadafc2c677c admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:00.225240 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db2aa3-98a4-49b2-8e3f-fadafc2c677c admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:00.225514 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db2aa3-98a4-49b2-8e3f-fadafc2c677c admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:00.226231 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39db2aa3-98a4-49b2-8e3f-fadafc2c677c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:00.228178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39db2aa3-98a4-49b2-8e3f-fadafc2c677c admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:00.270622 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-e27df099-466c-48f8-b752-07378b0232c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '915a1b3393bf4384a2d4d9a2bb8559e8'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:00.284514 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-e27df099-466c-48f8-b752-07378b0232c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '915a1b3393bf4384a2d4d9a2bb8559e8'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:00.292274 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-e27df099-466c-48f8-b752-07378b0232c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '915a1b3393bf4384a2d4d9a2bb8559e8'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:00.293396 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-e27df099-466c-48f8-b752-07378b0232c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '915a1b3393bf4384a2d4d9a2bb8559e8'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:00.318981 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1010/1974] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:14:00 2022] POST /identity/v3/users => generated 305 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:14:00.323135 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db2aa3-98a4-49b2-8e3f-fadafc2c677c admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:00.334706 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db2aa3-98a4-49b2-8e3f-fadafc2c677c admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:00.343356 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3049aa54-ff48-4239-8aac-d9bb2486f41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:00.344185 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049aa54-ff48-4239-8aac-d9bb2486f41e admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:00.344535 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049aa54-ff48-4239-8aac-d9bb2486f41e admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:00.344887 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3049aa54-ff48-4239-8aac-d9bb2486f41e admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:00.345684 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3049aa54-ff48-4239-8aac-d9bb2486f41e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:00.347375 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3049aa54-ff48-4239-8aac-d9bb2486f41e admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:00.358774 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1011/1975] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:14:00.364864 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3049aa54-ff48-4239-8aac-d9bb2486f41e admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:00.379338 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3049aa54-ff48-4239-8aac-d9bb2486f41e admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:00.380078 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-e27df099-466c-48f8-b752-07378b0232c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '915a1b3393bf4384a2d4d9a2bb8559e8'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:00.385076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9278467-42dc-416a-8d06-4fe3055bc963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:00.385753 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9278467-42dc-416a-8d06-4fe3055bc963 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:00.386040 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9278467-42dc-416a-8d06-4fe3055bc963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:00.386321 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9278467-42dc-416a-8d06-4fe3055bc963 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:00.387108 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 965/1976] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:14:00 2022] DELETE /identity/v3/users/915a1b3393bf4384a2d4d9a2bb8559e8 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:00.388282 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9278467-42dc-416a-8d06-4fe3055bc963 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:00.390568 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9278467-42dc-416a-8d06-4fe3055bc963 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:00.402585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1012/1977] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:14:00.406966 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27df099-466c-48f8-b752-07378b0232c1 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:00.408351 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e27df099-466c-48f8-b752-07378b0232c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:00.475243 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9278467-42dc-416a-8d06-4fe3055bc963 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:00.476992 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f9278467-42dc-416a-8d06-4fe3055bc963 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:00.484079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27df099-466c-48f8-b752-07378b0232c1 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:00.491074 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9278467-42dc-416a-8d06-4fe3055bc963 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:00.493719 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e11feac-d696-45f1-9b9e-d80c2032321d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:00.495009 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e11feac-d696-45f1-9b9e-d80c2032321d admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:00.495274 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e11feac-d696-45f1-9b9e-d80c2032321d admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:00.495561 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e11feac-d696-45f1-9b9e-d80c2032321d admin admin] PATH_INFO: `/v3/projects/b5664526f372472da13385845239c1a3/users/6c04736ff6a74cc3b228236efee80185/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:00.497184 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c6889b-5bb0-4b27-8b60-4a7978ffc1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:00.499248 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c6889b-5bb0-4b27-8b60-4a7978ffc1d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:00.499628 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c6889b-5bb0-4b27-8b60-4a7978ffc1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:00.500005 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c6889b-5bb0-4b27-8b60-4a7978ffc1d3 admin admin] PATH_INFO: `/v3/projects/550bd9c90de74021a41c06779ec2d136` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:00.501503 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c6889b-5bb0-4b27-8b60-4a7978ffc1d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=550bd9c90de74021a41c06779ec2d136)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:00.503862 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c6889b-5bb0-4b27-8b60-4a7978ffc1d3 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:00.538611 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e11feac-d696-45f1-9b9e-d80c2032321d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5664526f372472da13385845239c1a3, user_id=6c04736ff6a74cc3b228236efee80185, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:00.542238 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e11feac-d696-45f1-9b9e-d80c2032321d admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:00.611122 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 966/1978] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:14:00 2022] PUT /identity/v3/projects/b5664526f372472da13385845239c1a3/users/6c04736ff6a74cc3b228236efee80185/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:00.618151 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-22c6889b-5bb0-4b27-8b60-4a7978ffc1d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '550bd9c90de74021a41c06779ec2d136', 'request_id': 'req-22c6889b-5bb0-4b27-8b60-4a7978ffc1d3'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:00.621877 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943989b-ca1c-4efe-95fe-fb176514a399 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:00.622174 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943989b-ca1c-4efe-95fe-fb176514a399 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:00.622457 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f943989b-ca1c-4efe-95fe-fb176514a399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:00.627809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-22c6889b-5bb0-4b27-8b60-4a7978ffc1d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '550bd9c90de74021a41c06779ec2d136', 'request_id': 'req-22c6889b-5bb0-4b27-8b60-4a7978ffc1d3'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:00.633705 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-22c6889b-5bb0-4b27-8b60-4a7978ffc1d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '550bd9c90de74021a41c06779ec2d136', 'request_id': 'req-22c6889b-5bb0-4b27-8b60-4a7978ffc1d3'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:00.649963 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-f943989b-ca1c-4efe-95fe-fb176514a399 None None] MFA Rules not processed for user `6c04736ff6a74cc3b228236efee80185`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:00.651190 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-f943989b-ca1c-4efe-95fe-fb176514a399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:00.721819 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-22c6889b-5bb0-4b27-8b60-4a7978ffc1d3 admin admin] The token cache is being invalidate because project 550bd9c90de74021a41c06779ec2d136 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:14:00.721819 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-22c6889b-5bb0-4b27-8b60-4a7978ffc1d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:00.727020 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-22c6889b-5bb0-4b27-8b60-4a7978ffc1d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:00.727020 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1013/1979] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:14:00 2022] DELETE /identity/v3/projects/550bd9c90de74021a41c06779ec2d136 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:00.731316 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb64db-ece6-49d5-887e-bf0d56c42643 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:00.731650 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb64db-ece6-49d5-887e-bf0d56c42643 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:00.731966 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bb64db-ece6-49d5-887e-bf0d56c42643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:00.743459 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:00.743814 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 967/1980] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:00 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:00.746753 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f943989b-ca1c-4efe-95fe-fb176514a399 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:00.753168 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-48bb64db-ece6-49d5-887e-bf0d56c42643 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:00.754732 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-48bb64db-ece6-49d5-887e-bf0d56c42643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:00.829970 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1014/1981] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:00.831416 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a3614-1040-44ce-ab88-59bff3cac420 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:00.831689 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a3614-1040-44ce-ab88-59bff3cac420 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:00.831992 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0a3614-1040-44ce-ab88-59bff3cac420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:00.856866 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f943989b-ca1c-4efe-95fe-fb176514a399 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:00.859611 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-ad0a3614-1040-44ce-ab88-59bff3cac420 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:00.860851 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-ad0a3614-1040-44ce-ab88-59bff3cac420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:00.862220 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f909a5-6531-4437-9a09-9dbf6dbb65ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:00.864461 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f909a5-6531-4437-9a09-9dbf6dbb65ae admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:00.864461 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f909a5-6531-4437-9a09-9dbf6dbb65ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:00.864461 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f909a5-6531-4437-9a09-9dbf6dbb65ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:00.864715 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f909a5-6531-4437-9a09-9dbf6dbb65ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:00.867610 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f909a5-6531-4437-9a09-9dbf6dbb65ae admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:00.872335 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1015/1982] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:14:00.876857 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a3614-1040-44ce-ab88-59bff3cac420 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:00.886647 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0a3614-1040-44ce-ab88-59bff3cac420 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:00.892175 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d753ee-d788-495d-9d46-b599d9724b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:00.892858 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d753ee-d788-495d-9d46-b599d9724b14 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:00.893145 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d753ee-d788-495d-9d46-b599d9724b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:00.893424 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d753ee-d788-495d-9d46-b599d9724b14 admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:00.894148 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d753ee-d788-495d-9d46-b599d9724b14 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:14:00.894457 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d753ee-d788-495d-9d46-b599d9724b14 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:00.896104 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d753ee-d788-495d-9d46-b599d9724b14 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:00.909293 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1016/1983] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:14:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:14:00.925673 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d753ee-d788-495d-9d46-b599d9724b14 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:00.937586 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d753ee-d788-495d-9d46-b599d9724b14 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:00.943041 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d22cd86-c6f0-4766-9fa1-fc47bb2c7334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:00.943704 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d22cd86-c6f0-4766-9fa1-fc47bb2c7334 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:00.944067 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d22cd86-c6f0-4766-9fa1-fc47bb2c7334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:00.944389 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d22cd86-c6f0-4766-9fa1-fc47bb2c7334 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:00.945123 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d22cd86-c6f0-4766-9fa1-fc47bb2c7334 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:00.947329 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d22cd86-c6f0-4766-9fa1-fc47bb2c7334 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:01.147423 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 968/1984] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:14:00 2022] POST /identity/v3/projects => generated 407 bytes in 400 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:14:01.147423 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f909a5-6531-4437-9a09-9dbf6dbb65ae admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:01.157344 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-79f909a5-6531-4437-9a09-9dbf6dbb65ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:01.211460 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f909a5-6531-4437-9a09-9dbf6dbb65ae admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:01.216025 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47425c85-0a7d-4caa-a57a-0ea9c5b5129c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:01.220209 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47425c85-0a7d-4caa-a57a-0ea9c5b5129c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:01.220209 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47425c85-0a7d-4caa-a57a-0ea9c5b5129c admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:01.220209 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47425c85-0a7d-4caa-a57a-0ea9c5b5129c admin admin] PATH_INFO: `/v3/projects/d7d7dbfdc0774456b89d2d17b4f74f7e` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:01.220209 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47425c85-0a7d-4caa-a57a-0ea9c5b5129c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7d7dbfdc0774456b89d2d17b4f74f7e)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:01.220837 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47425c85-0a7d-4caa-a57a-0ea9c5b5129c admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:01.253311 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1017/1985] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:14:00 2022] POST /identity/v3/projects => generated 373 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:14:01.253311 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d22cd86-c6f0-4766-9fa1-fc47bb2c7334 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:01.253311 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-5d22cd86-c6f0-4766-9fa1-fc47bb2c7334 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:01.338216 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d22cd86-c6f0-4766-9fa1-fc47bb2c7334 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:01.348460 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea205fcd-9fee-44a3-b640-cc731c778112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:01.349233 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea205fcd-9fee-44a3-b640-cc731c778112 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:01.349587 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea205fcd-9fee-44a3-b640-cc731c778112 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:01.349929 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea205fcd-9fee-44a3-b640-cc731c778112 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:01.350813 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea205fcd-9fee-44a3-b640-cc731c778112 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:01.353281 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea205fcd-9fee-44a3-b640-cc731c778112 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:01.373524 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-47425c85-0a7d-4caa-a57a-0ea9c5b5129c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7d7dbfdc0774456b89d2d17b4f74f7e', 'request_id': 'req-47425c85-0a7d-4caa-a57a-0ea9c5b5129c'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:01.379949 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-47425c85-0a7d-4caa-a57a-0ea9c5b5129c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7d7dbfdc0774456b89d2d17b4f74f7e', 'request_id': 'req-47425c85-0a7d-4caa-a57a-0ea9c5b5129c'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:01.452664 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1018/1986] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:14:01 2022] POST /identity/v3/users => generated 310 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:14:01.453020 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea205fcd-9fee-44a3-b640-cc731c778112 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:01.463157 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea205fcd-9fee-44a3-b640-cc731c778112 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:01.505804 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-47425c85-0a7d-4caa-a57a-0ea9c5b5129c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7d7dbfdc0774456b89d2d17b4f74f7e', 'request_id': 'req-47425c85-0a7d-4caa-a57a-0ea9c5b5129c'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:01.521386 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-47425c85-0a7d-4caa-a57a-0ea9c5b5129c admin admin] The token cache is being invalidate because project d7d7dbfdc0774456b89d2d17b4f74f7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:14:01.522031 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-47425c85-0a7d-4caa-a57a-0ea9c5b5129c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:01.522675 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-47425c85-0a7d-4caa-a57a-0ea9c5b5129c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:01.525179 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 969/1987] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:14:01 2022] DELETE /identity/v3/projects/d7d7dbfdc0774456b89d2d17b4f74f7e => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:01.525978 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47425c85-0a7d-4caa-a57a-0ea9c5b5129c admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:01.528149 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-47425c85-0a7d-4caa-a57a-0ea9c5b5129c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:01.537344 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452f9f2c-d831-4197-b118-c2b464be206f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:01.537344 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f9f2c-d831-4197-b118-c2b464be206f admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:01.537344 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f9f2c-d831-4197-b118-c2b464be206f admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:01.537344 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452f9f2c-d831-4197-b118-c2b464be206f admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:01.537771 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452f9f2c-d831-4197-b118-c2b464be206f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:01.537771 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452f9f2c-d831-4197-b118-c2b464be206f admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:01.542980 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47425c85-0a7d-4caa-a57a-0ea9c5b5129c admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:01.549339 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26864fde-354b-49fa-b5a7-61472a99cd19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:01.550788 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26864fde-354b-49fa-b5a7-61472a99cd19 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:01.550788 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26864fde-354b-49fa-b5a7-61472a99cd19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:01.551104 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26864fde-354b-49fa-b5a7-61472a99cd19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:01.551973 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26864fde-354b-49fa-b5a7-61472a99cd19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:01.553615 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26864fde-354b-49fa-b5a7-61472a99cd19 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:01.575441 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 970/1988] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:14:01.575441 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea3a69-bfbd-4ba3-94a3-84fa77b8941c None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:01.575441 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea3a69-bfbd-4ba3-94a3-84fa77b8941c None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:01.575441 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ea3a69-bfbd-4ba3-94a3-84fa77b8941c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:01.594917 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-e2ea3a69-bfbd-4ba3-94a3-84fa77b8941c None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:01.596118 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e2ea3a69-bfbd-4ba3-94a3-84fa77b8941c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:01.605346 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 971/1989] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:01.606417 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ea3a69-bfbd-4ba3-94a3-84fa77b8941c None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:01.618856 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ea3a69-bfbd-4ba3-94a3-84fa77b8941c None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:01.626845 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af2c74-6803-4386-aee9-edeb04ef2fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:01.628848 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af2c74-6803-4386-aee9-edeb04ef2fff admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:01.629447 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af2c74-6803-4386-aee9-edeb04ef2fff admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:01.629915 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9af2c74-6803-4386-aee9-edeb04ef2fff admin admin] PATH_INFO: `/v3/projects/069c8384756f4796a71f5df99f15a80f/users/3d7b125a10cb4ad889fd3b72b13ccdf9/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:01.645172 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1019/1990] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:14:01 2022] POST /identity/v3/users => generated 287 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:14:01.647103 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801ff41-677d-4070-b788-9fba92e37231 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:01.647499 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801ff41-677d-4070-b788-9fba92e37231 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:01.648169 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b801ff41-677d-4070-b788-9fba92e37231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:01.660146 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9af2c74-6803-4386-aee9-edeb04ef2fff admin admin] RBAC: Authorizing `identity:create_grant(project_id=069c8384756f4796a71f5df99f15a80f, user_id=3d7b125a10cb4ad889fd3b72b13ccdf9, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:01.662864 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9af2c74-6803-4386-aee9-edeb04ef2fff admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:01.668155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-b801ff41-677d-4070-b788-9fba92e37231 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:01.669576 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b801ff41-677d-4070-b788-9fba92e37231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:01.678846 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1020/1991] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:01.679879 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b801ff41-677d-4070-b788-9fba92e37231 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:01.681872 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b801ff41-677d-4070-b788-9fba92e37231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:01.695379 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b801ff41-677d-4070-b788-9fba92e37231 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:01.700927 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ae12a-876d-4d1f-b7f2-00584961884d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:01.701607 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ae12a-876d-4d1f-b7f2-00584961884d admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:01.701917 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ae12a-876d-4d1f-b7f2-00584961884d admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:01.702197 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ae12a-876d-4d1f-b7f2-00584961884d admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:01.703134 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ae12a-876d-4d1f-b7f2-00584961884d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:01.708152 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ae12a-876d-4d1f-b7f2-00584961884d admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:01.712175 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 972/1992] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:14:01 2022] PUT /identity/v3/projects/069c8384756f4796a71f5df99f15a80f/users/3d7b125a10cb4ad889fd3b72b13ccdf9/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:01.713138 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af2c74-6803-4386-aee9-edeb04ef2fff admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:01.722287 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1021/1993] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:14:01.723581 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ae12a-876d-4d1f-b7f2-00584961884d admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:01.741610 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ae12a-876d-4d1f-b7f2-00584961884d admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:01.809356 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e491652a-17fd-4631-bb69-c75a2cfc4c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:01.810058 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e491652a-17fd-4631-bb69-c75a2cfc4c48 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:01.810401 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e491652a-17fd-4631-bb69-c75a2cfc4c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:01.811341 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e491652a-17fd-4631-bb69-c75a2cfc4c48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:01.812113 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e491652a-17fd-4631-bb69-c75a2cfc4c48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:01.817916 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e491652a-17fd-4631-bb69-c75a2cfc4c48 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:01.818052 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9af2c74-6803-4386-aee9-edeb04ef2fff admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:01.821832 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04121b1-9792-4b11-80b1-9375db5677c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:01.823096 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04121b1-9792-4b11-80b1-9375db5677c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:01.823520 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04121b1-9792-4b11-80b1-9375db5677c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:01.823993 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04121b1-9792-4b11-80b1-9375db5677c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:01.824574 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04121b1-9792-4b11-80b1-9375db5677c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:14:01.824887 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04121b1-9792-4b11-80b1-9375db5677c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:01.826654 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1022/1994] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:14:01.827582 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e491652a-17fd-4631-bb69-c75a2cfc4c48 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:01.830653 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04121b1-9792-4b11-80b1-9375db5677c3 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:01.855218 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 973/1995] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:14:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:14:01.855218 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04121b1-9792-4b11-80b1-9375db5677c3 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:01.861180 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e491652a-17fd-4631-bb69-c75a2cfc4c48 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:01.864611 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04121b1-9792-4b11-80b1-9375db5677c3 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:01.874513 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002bc430-7d78-4673-b7a1-737025d8a248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:01.875200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002bc430-7d78-4673-b7a1-737025d8a248 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:01.875489 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002bc430-7d78-4673-b7a1-737025d8a248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:01.875803 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002bc430-7d78-4673-b7a1-737025d8a248 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:01.876532 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002bc430-7d78-4673-b7a1-737025d8a248 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:01.878140 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002bc430-7d78-4673-b7a1-737025d8a248 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:01.879961 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac541c-a2f7-43ee-983d-9a1a7c46e831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:01.880871 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac541c-a2f7-43ee-983d-9a1a7c46e831 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:01.881156 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac541c-a2f7-43ee-983d-9a1a7c46e831 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:01.881433 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac541c-a2f7-43ee-983d-9a1a7c46e831 admin admin] PATH_INFO: `/v3/projects/069c8384756f4796a71f5df99f15a80f/users/3d7b125a10cb4ad889fd3b72b13ccdf9/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:01.883655 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac541c-a2f7-43ee-983d-9a1a7c46e831 admin admin] RBAC: Authorizing `identity:create_grant(project_id=069c8384756f4796a71f5df99f15a80f, user_id=3d7b125a10cb4ad889fd3b72b13ccdf9, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:01.886774 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac541c-a2f7-43ee-983d-9a1a7c46e831 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:01.890115 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1023/1996] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:01 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:14:01.891157 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002bc430-7d78-4673-b7a1-737025d8a248 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:01.904543 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002bc430-7d78-4673-b7a1-737025d8a248 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:01.907822 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 974/1997] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:14:01 2022] PUT /identity/v3/projects/069c8384756f4796a71f5df99f15a80f/users/3d7b125a10cb4ad889fd3b72b13ccdf9/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:01.908755 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac541c-a2f7-43ee-983d-9a1a7c46e831 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:02.000674 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d023979-04e6-4aea-86bd-9f389f53a2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:02.001358 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d023979-04e6-4aea-86bd-9f389f53a2a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:02.001648 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d023979-04e6-4aea-86bd-9f389f53a2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:02.001929 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d023979-04e6-4aea-86bd-9f389f53a2a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:02.002194 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac541c-a2f7-43ee-983d-9a1a7c46e831 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:02.003176 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d023979-04e6-4aea-86bd-9f389f53a2a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:02.005517 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d023979-04e6-4aea-86bd-9f389f53a2a8 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:02.008607 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca5345-b17c-4eea-abcc-d9e4fdd91015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:02.010006 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca5345-b17c-4eea-abcc-d9e4fdd91015 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:02.010412 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca5345-b17c-4eea-abcc-d9e4fdd91015 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:02.010889 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca5345-b17c-4eea-abcc-d9e4fdd91015 admin admin] PATH_INFO: `/v3/projects/cb46b27002d34dedb73a392f32c0c91c/users/c42836dfcbcb485c9fd5310aaebb5ca9/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:02.031041 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beca5345-b17c-4eea-abcc-d9e4fdd91015 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb46b27002d34dedb73a392f32c0c91c, user_id=c42836dfcbcb485c9fd5310aaebb5ca9, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:02.033868 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beca5345-b17c-4eea-abcc-d9e4fdd91015 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:02.110410 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1024/1998] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:14:01 2022] POST /identity/v3/projects => generated 381 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:14:02.111953 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6cf28-947d-494f-a450-b2237ff1bf92 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:02.112250 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6cf28-947d-494f-a450-b2237ff1bf92 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:02.112532 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a6cf28-947d-494f-a450-b2237ff1bf92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:02.148490 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 975/1999] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:14:01 2022] PUT /identity/v3/projects/cb46b27002d34dedb73a392f32c0c91c/users/c42836dfcbcb485c9fd5310aaebb5ca9/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:02.149307 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca5345-b17c-4eea-abcc-d9e4fdd91015 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:02.177167 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-b7a6cf28-947d-494f-a450-b2237ff1bf92 None None] MFA Rules not processed for user `3d7b125a10cb4ad889fd3b72b13ccdf9`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:02.178511 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b7a6cf28-947d-494f-a450-b2237ff1bf92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:02.245109 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca5345-b17c-4eea-abcc-d9e4fdd91015 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:02.250633 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e800e49-7185-4229-8822-86e9bf922aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:02.251340 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e800e49-7185-4229-8822-86e9bf922aab admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:02.251623 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e800e49-7185-4229-8822-86e9bf922aab admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:02.251932 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e800e49-7185-4229-8822-86e9bf922aab admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:02.252612 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e800e49-7185-4229-8822-86e9bf922aab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:02.254562 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e800e49-7185-4229-8822-86e9bf922aab admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:02.256565 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1025/2000] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:02 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:02.258079 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6c89a-d23e-42d8-8258-3ac3adc799dc None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:02.258381 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6c89a-d23e-42d8-8258-3ac3adc799dc None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:02.258669 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6c89a-d23e-42d8-8258-3ac3adc799dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:02.287238 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-f7a6c89a-d23e-42d8-8258-3ac3adc799dc None None] MFA Rules not processed for user `c42836dfcbcb485c9fd5310aaebb5ca9`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:02.288503 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f7a6c89a-d23e-42d8-8258-3ac3adc799dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:02.356921 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1026/2001] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:02 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:02.357899 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a6c89a-d23e-42d8-8258-3ac3adc799dc None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:02.359295 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f7a6c89a-d23e-42d8-8258-3ac3adc799dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:02.372968 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 976/2002] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:14:02 2022] POST /identity/v3/users => generated 291 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:14:02.376979 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e800e49-7185-4229-8822-86e9bf922aab admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:02.389724 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e800e49-7185-4229-8822-86e9bf922aab admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:02.394687 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814ad7d8-cbbe-42c0-8a46-dc5ffbf7b8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:02.395973 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ad7d8-cbbe-42c0-8a46-dc5ffbf7b8cb admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:02.396475 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ad7d8-cbbe-42c0-8a46-dc5ffbf7b8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:02.396952 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814ad7d8-cbbe-42c0-8a46-dc5ffbf7b8cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:02.397635 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814ad7d8-cbbe-42c0-8a46-dc5ffbf7b8cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:02.399763 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814ad7d8-cbbe-42c0-8a46-dc5ffbf7b8cb admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:02.419312 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 977/2003] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:14:02.424565 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814ad7d8-cbbe-42c0-8a46-dc5ffbf7b8cb admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:02.429193 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a6c89a-d23e-42d8-8258-3ac3adc799dc None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:02.432921 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f89fbd-e085-4d4a-8cf6-63a093c377e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:02.433577 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f89fbd-e085-4d4a-8cf6-63a093c377e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:02.433865 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f89fbd-e085-4d4a-8cf6-63a093c377e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:02.434175 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f89fbd-e085-4d4a-8cf6-63a093c377e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:02.435262 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814ad7d8-cbbe-42c0-8a46-dc5ffbf7b8cb admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:02.440174 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7521565f-d894-4293-9e70-82e0b24d844c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:02.440863 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7521565f-d894-4293-9e70-82e0b24d844c admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:02.441378 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7521565f-d894-4293-9e70-82e0b24d844c admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:02.441644 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7521565f-d894-4293-9e70-82e0b24d844c admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:02.442294 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7521565f-d894-4293-9e70-82e0b24d844c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:02.444023 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7521565f-d894-4293-9e70-82e0b24d844c admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:02.445619 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f89fbd-e085-4d4a-8cf6-63a093c377e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:02.447195 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f89fbd-e085-4d4a-8cf6-63a093c377e2 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:02.457396 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 978/2004] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:14:02.458199 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1027/2005] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:14:02 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:14:02.463307 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7521565f-d894-4293-9e70-82e0b24d844c admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:02.474465 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7521565f-d894-4293-9e70-82e0b24d844c admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:02.482721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4128b18-893b-43d9-9519-141a749bebf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:02.482721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4128b18-893b-43d9-9519-141a749bebf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:02.482721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4128b18-893b-43d9-9519-141a749bebf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:02.482721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4128b18-893b-43d9-9519-141a749bebf2 admin admin] PATH_INFO: `/v3/projects/bbb4139cfb67483b935200ba0d9d3ba0/users/452219fec55e4c6abf58e7fa4d91ac5f/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:02.498765 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4128b18-893b-43d9-9519-141a749bebf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbb4139cfb67483b935200ba0d9d3ba0, user_id=452219fec55e4c6abf58e7fa4d91ac5f, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:02.498937 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f89fbd-e085-4d4a-8cf6-63a093c377e2 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:02.500293 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-e8f89fbd-e085-4d4a-8cf6-63a093c377e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:02.501641 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4128b18-893b-43d9-9519-141a749bebf2 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:02.511809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f89fbd-e085-4d4a-8cf6-63a093c377e2 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:02.516163 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc25c9-16ee-45a4-8d34-54d38b825160 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:02.520620 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc25c9-16ee-45a4-8d34-54d38b825160 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:02.520620 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc25c9-16ee-45a4-8d34-54d38b825160 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:02.520620 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dc25c9-16ee-45a4-8d34-54d38b825160 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:02.526549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dc25c9-16ee-45a4-8d34-54d38b825160 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:02.532503 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5dc25c9-16ee-45a4-8d34-54d38b825160 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:02.542259 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1028/2006] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:14:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:14:02.567934 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc25c9-16ee-45a4-8d34-54d38b825160 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:02.575233 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 979/2007] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:14:02 2022] PUT /identity/v3/projects/bbb4139cfb67483b935200ba0d9d3ba0/users/452219fec55e4c6abf58e7fa4d91ac5f/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:02.579861 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dc25c9-16ee-45a4-8d34-54d38b825160 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:02.587279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f942eab-07fb-4630-ad25-5402cebb1ac4 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:02.587578 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f942eab-07fb-4630-ad25-5402cebb1ac4 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:02.587911 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f942eab-07fb-4630-ad25-5402cebb1ac4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:02.616761 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-9f942eab-07fb-4630-ad25-5402cebb1ac4 None None] MFA Rules not processed for user `452219fec55e4c6abf58e7fa4d91ac5f`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:02.617917 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-9f942eab-07fb-4630-ad25-5402cebb1ac4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:02.643387 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb097c96-17f1-4854-bc42-4f5abaca6099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:02.644438 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb097c96-17f1-4854-bc42-4f5abaca6099 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:02.644791 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb097c96-17f1-4854-bc42-4f5abaca6099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:02.645136 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb097c96-17f1-4854-bc42-4f5abaca6099 admin admin] PATH_INFO: `/v3/users/c42836dfcbcb485c9fd5310aaebb5ca9` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:02.646579 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb097c96-17f1-4854-bc42-4f5abaca6099 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c42836dfcbcb485c9fd5310aaebb5ca9)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:02.648789 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb097c96-17f1-4854-bc42-4f5abaca6099 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:02.705783 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 980/2008] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:14:02.710276 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f942eab-07fb-4630-ad25-5402cebb1ac4 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:02.723048 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f942eab-07fb-4630-ad25-5402cebb1ac4 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:02.728934 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598534b1-7144-4746-869e-13c423677911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:02.729615 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598534b1-7144-4746-869e-13c423677911 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:02.729906 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598534b1-7144-4746-869e-13c423677911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:02.730186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598534b1-7144-4746-869e-13c423677911 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:02.730952 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598534b1-7144-4746-869e-13c423677911 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:02.733892 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598534b1-7144-4746-869e-13c423677911 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:02.828174 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-cb097c96-17f1-4854-bc42-4f5abaca6099 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c42836dfcbcb485c9fd5310aaebb5ca9'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:02.837567 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-cb097c96-17f1-4854-bc42-4f5abaca6099 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c42836dfcbcb485c9fd5310aaebb5ca9'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:02.838350 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-cb097c96-17f1-4854-bc42-4f5abaca6099 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c42836dfcbcb485c9fd5310aaebb5ca9'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:02.840130 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 981/2009] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:14:02 2022] POST /identity/v3/projects => generated 383 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:14:02.844924 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598534b1-7144-4746-869e-13c423677911 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:02.856344 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-cb097c96-17f1-4854-bc42-4f5abaca6099 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c42836dfcbcb485c9fd5310aaebb5ca9'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:02.900020 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:02.908939 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598534b1-7144-4746-869e-13c423677911 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:02.913579 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c5b9e-318e-4ab2-ac9e-a099b0a9f6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:02.914246 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c5b9e-318e-4ab2-ac9e-a099b0a9f6a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:02.914532 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c5b9e-318e-4ab2-ac9e-a099b0a9f6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:02.914809 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c5b9e-318e-4ab2-ac9e-a099b0a9f6a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:02.915504 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c5b9e-318e-4ab2-ac9e-a099b0a9f6a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:02.916564 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-cb097c96-17f1-4854-bc42-4f5abaca6099 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c42836dfcbcb485c9fd5310aaebb5ca9'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:02.917452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c5b9e-318e-4ab2-ac9e-a099b0a9f6a3 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:02.923867 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1029/2010] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:14:02 2022] DELETE /identity/v3/users/c42836dfcbcb485c9fd5310aaebb5ca9 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:14:02.929934 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb097c96-17f1-4854-bc42-4f5abaca6099 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:02.929934 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-cb097c96-17f1-4854-bc42-4f5abaca6099 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:02.984847 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb097c96-17f1-4854-bc42-4f5abaca6099 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:02.987758 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 982/2011] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:14:02 2022] POST /identity/v3/users => generated 298 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:14:02.987758 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eeee57-b689-4b69-8a25-21137676474f None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:02.987758 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eeee57-b689-4b69-8a25-21137676474f None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:02.988063 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7eeee57-b689-4b69-8a25-21137676474f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:02.992129 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dcbbdd-dbf4-44a7-a347-72e0fc598e87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:02.993252 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dcbbdd-dbf4-44a7-a347-72e0fc598e87 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:02.993252 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dcbbdd-dbf4-44a7-a347-72e0fc598e87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:02.993435 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dcbbdd-dbf4-44a7-a347-72e0fc598e87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:02.995205 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-72dcbbdd-dbf4-44a7-a347-72e0fc598e87 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:03.008321 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-e7eeee57-b689-4b69-8a25-21137676474f None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:03.009531 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e7eeee57-b689-4b69-8a25-21137676474f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:03.059020 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dcbbdd-dbf4-44a7-a347-72e0fc598e87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:03.060609 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dcbbdd-dbf4-44a7-a347-72e0fc598e87 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:03.070822 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1030/2012] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:14:02 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:14:03.071867 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dcbbdd-dbf4-44a7-a347-72e0fc598e87 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:03.085920 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 983/2013] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:03.086990 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-72dcbbdd-dbf4-44a7-a347-72e0fc598e87 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:03.087084 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eeee57-b689-4b69-8a25-21137676474f None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:03.087084 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e7eeee57-b689-4b69-8a25-21137676474f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:03.095979 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dcbbdd-dbf4-44a7-a347-72e0fc598e87 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:03.101087 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f868acd-9bfc-449a-aa70-7f6046738a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:03.101776 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f868acd-9bfc-449a-aa70-7f6046738a09 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:03.102163 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f868acd-9bfc-449a-aa70-7f6046738a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:03.102809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f868acd-9bfc-449a-aa70-7f6046738a09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:03.103621 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f868acd-9bfc-449a-aa70-7f6046738a09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:03.106600 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f868acd-9bfc-449a-aa70-7f6046738a09 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:03.120689 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1031/2014] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:03 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:14:03.122447 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f868acd-9bfc-449a-aa70-7f6046738a09 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:03.133818 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f868acd-9bfc-449a-aa70-7f6046738a09 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:03.139247 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e33be-3e8f-4f09-b40b-2cac1c906ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:03.140177 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e33be-3e8f-4f09-b40b-2cac1c906ee9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:03.140459 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e33be-3e8f-4f09-b40b-2cac1c906ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:03.140742 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e33be-3e8f-4f09-b40b-2cac1c906ee9 admin admin] PATH_INFO: `/v3/projects/134c4d05e49045adb315afaaea428025/users/da89e0b6c89e4c0bb7008c69d7d74949/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:03.153933 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:03.160261 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8e33be-3e8f-4f09-b40b-2cac1c906ee9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=134c4d05e49045adb315afaaea428025, user_id=da89e0b6c89e4c0bb7008c69d7d74949, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:03.164054 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8e33be-3e8f-4f09-b40b-2cac1c906ee9 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:03.172890 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7eeee57-b689-4b69-8a25-21137676474f None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:03.177944 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411333d5-7cc5-44f3-8a6a-725bc20359c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:03.178878 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411333d5-7cc5-44f3-8a6a-725bc20359c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:03.178993 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411333d5-7cc5-44f3-8a6a-725bc20359c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:03.179202 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411333d5-7cc5-44f3-8a6a-725bc20359c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:03.191298 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411333d5-7cc5-44f3-8a6a-725bc20359c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:03.193298 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411333d5-7cc5-44f3-8a6a-725bc20359c9 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:03.208222 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 984/2015] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:14:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:14:03.238895 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1032/2016] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:14:03 2022] PUT /identity/v3/projects/134c4d05e49045adb315afaaea428025/users/da89e0b6c89e4c0bb7008c69d7d74949/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:03.242749 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411333d5-7cc5-44f3-8a6a-725bc20359c9 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:03.255215 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411333d5-7cc5-44f3-8a6a-725bc20359c9 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:03.346477 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76023a5b-6eb8-455a-a401-49b81a326995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:03.347161 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76023a5b-6eb8-455a-a401-49b81a326995 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:03.347445 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76023a5b-6eb8-455a-a401-49b81a326995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:03.347720 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76023a5b-6eb8-455a-a401-49b81a326995 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:03.348474 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76023a5b-6eb8-455a-a401-49b81a326995 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:03.350192 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76023a5b-6eb8-455a-a401-49b81a326995 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:03.361133 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 985/2017] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:03 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:14:03.365571 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e33be-3e8f-4f09-b40b-2cac1c906ee9 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:03.377230 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e33be-3e8f-4f09-b40b-2cac1c906ee9 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:03.382936 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b4b99-0d4c-44e6-af01-5df98354eb12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:03.386434 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b4b99-0d4c-44e6-af01-5df98354eb12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:03.386434 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b4b99-0d4c-44e6-af01-5df98354eb12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:03.386434 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b4b99-0d4c-44e6-af01-5df98354eb12 admin admin] PATH_INFO: `/v3/projects/134c4d05e49045adb315afaaea428025/users/da89e0b6c89e4c0bb7008c69d7d74949/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:03.386690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016b4b99-0d4c-44e6-af01-5df98354eb12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=134c4d05e49045adb315afaaea428025, user_id=da89e0b6c89e4c0bb7008c69d7d74949, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:03.390251 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016b4b99-0d4c-44e6-af01-5df98354eb12 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:03.408806 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1033/2018] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:14:03 2022] PUT /identity/v3/projects/134c4d05e49045adb315afaaea428025/users/da89e0b6c89e4c0bb7008c69d7d74949/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:03.419502 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e51c94-febd-4c9f-b6ca-ef761c63738f None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:03.420127 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e51c94-febd-4c9f-b6ca-ef761c63738f None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:03.420127 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e51c94-febd-4c9f-b6ca-ef761c63738f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:03.449940 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-73e51c94-febd-4c9f-b6ca-ef761c63738f None None] MFA Rules not processed for user `da89e0b6c89e4c0bb7008c69d7d74949`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:03.451259 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-73e51c94-febd-4c9f-b6ca-ef761c63738f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:03.530287 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 986/2019] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:03 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:03.542033 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b4b99-0d4c-44e6-af01-5df98354eb12 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:03.544777 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-016b4b99-0d4c-44e6-af01-5df98354eb12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:03.609937 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016b4b99-0d4c-44e6-af01-5df98354eb12 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:03.633549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d900ad5c-bf65-495b-8372-da9891fa1964 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:03.633549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900ad5c-bf65-495b-8372-da9891fa1964 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:03.633549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900ad5c-bf65-495b-8372-da9891fa1964 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:03.633549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900ad5c-bf65-495b-8372-da9891fa1964 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:03.653158 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d900ad5c-bf65-495b-8372-da9891fa1964 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:03.655533 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d900ad5c-bf65-495b-8372-da9891fa1964 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:03.668337 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1034/2020] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:14:03 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:14:03.850414 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e51c94-febd-4c9f-b6ca-ef761c63738f None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:03.852669 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-73e51c94-febd-4c9f-b6ca-ef761c63738f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:03.957594 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:03.967547 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e51c94-febd-4c9f-b6ca-ef761c63738f None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:03.973425 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84799c61-1dfb-4811-ad37-175296fdee41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:03.974603 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84799c61-1dfb-4811-ad37-175296fdee41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:03.974899 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84799c61-1dfb-4811-ad37-175296fdee41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:03.975181 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84799c61-1dfb-4811-ad37-175296fdee41 admin admin] PATH_INFO: `/v3/projects/cb46b27002d34dedb73a392f32c0c91c` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:03.976455 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84799c61-1dfb-4811-ad37-175296fdee41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb46b27002d34dedb73a392f32c0c91c)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:03.978932 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84799c61-1dfb-4811-ad37-175296fdee41 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:04.020202 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d900ad5c-bf65-495b-8372-da9891fa1964 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:04.044164 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d900ad5c-bf65-495b-8372-da9891fa1964 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:04.049250 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ac8017-5020-44d9-92ec-4fdc61fb4607 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:04.049983 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac8017-5020-44d9-92ec-4fdc61fb4607 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:04.050183 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac8017-5020-44d9-92ec-4fdc61fb4607 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:04.050455 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ac8017-5020-44d9-92ec-4fdc61fb4607 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:04.053420 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f3ac8017-5020-44d9-92ec-4fdc61fb4607 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:04.118245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ac8017-5020-44d9-92ec-4fdc61fb4607 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:04.119693 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ac8017-5020-44d9-92ec-4fdc61fb4607 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:04.130174 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1035/2021] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:14:04 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:14:04.141794 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-84799c61-1dfb-4811-ad37-175296fdee41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb46b27002d34dedb73a392f32c0c91c', 'request_id': 'req-84799c61-1dfb-4811-ad37-175296fdee41'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:04.148113 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-84799c61-1dfb-4811-ad37-175296fdee41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb46b27002d34dedb73a392f32c0c91c', 'request_id': 'req-84799c61-1dfb-4811-ad37-175296fdee41'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:04.237234 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-84799c61-1dfb-4811-ad37-175296fdee41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb46b27002d34dedb73a392f32c0c91c', 'request_id': 'req-84799c61-1dfb-4811-ad37-175296fdee41'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:04.248609 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-84799c61-1dfb-4811-ad37-175296fdee41 admin admin] The token cache is being invalidate because project cb46b27002d34dedb73a392f32c0c91c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:14:04.248903 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-84799c61-1dfb-4811-ad37-175296fdee41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:04.249539 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-84799c61-1dfb-4811-ad37-175296fdee41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:04.251102 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 987/2022] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:14:03 2022] DELETE /identity/v3/projects/cb46b27002d34dedb73a392f32c0c91c => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:04.256076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61085bf-7b97-4336-a292-8f9ceb07771a None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:04.256367 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61085bf-7b97-4336-a292-8f9ceb07771a None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:04.256643 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61085bf-7b97-4336-a292-8f9ceb07771a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:04.280560 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-d61085bf-7b97-4336-a292-8f9ceb07771a None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:04.281775 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d61085bf-7b97-4336-a292-8f9ceb07771a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:04.347119 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1036/2023] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:14:04.354874 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc684a-1edd-41b4-bf91-eb8dbc4c5f15 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:04.355170 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc684a-1edd-41b4-bf91-eb8dbc4c5f15 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:04.355573 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc684a-1edd-41b4-bf91-eb8dbc4c5f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:04.375114 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-9acc684a-1edd-41b4-bf91-eb8dbc4c5f15 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:04.376351 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-9acc684a-1edd-41b4-bf91-eb8dbc4c5f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:04.389130 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 988/2024] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:04.403528 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61085bf-7b97-4336-a292-8f9ceb07771a None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:04.415195 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc684a-1edd-41b4-bf91-eb8dbc4c5f15 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:04.419424 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-9acc684a-1edd-41b4-bf91-eb8dbc4c5f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:04.432253 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61085bf-7b97-4336-a292-8f9ceb07771a None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:04.438424 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc684a-1edd-41b4-bf91-eb8dbc4c5f15 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:04.443534 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74753d66-0794-4f27-b545-ef1a8320d1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:04.448335 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74753d66-0794-4f27-b545-ef1a8320d1a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:04.448335 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74753d66-0794-4f27-b545-ef1a8320d1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:04.448335 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74753d66-0794-4f27-b545-ef1a8320d1a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:04.448335 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74753d66-0794-4f27-b545-ef1a8320d1a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:14:04.448335 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74753d66-0794-4f27-b545-ef1a8320d1a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:04.448335 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74753d66-0794-4f27-b545-ef1a8320d1a3 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:04.451568 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075ac002-6b03-47df-9887-ea83a9cd0614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:04.452622 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ac002-6b03-47df-9887-ea83a9cd0614 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:04.452906 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ac002-6b03-47df-9887-ea83a9cd0614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:04.453183 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075ac002-6b03-47df-9887-ea83a9cd0614 admin admin] PATH_INFO: `/v3/users/b0d3a6765ea9469ebd3e526f54904803` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:04.455265 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ac002-6b03-47df-9887-ea83a9cd0614 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0d3a6765ea9469ebd3e526f54904803)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:04.457200 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075ac002-6b03-47df-9887-ea83a9cd0614 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:04.469715 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1037/2025] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:14:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:14:04.470975 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74753d66-0794-4f27-b545-ef1a8320d1a3 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:04.482188 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74753d66-0794-4f27-b545-ef1a8320d1a3 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:04.488247 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e3e7b9-0a37-4568-9838-cf45fa291f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:04.489193 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e3e7b9-0a37-4568-9838-cf45fa291f31 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:04.489699 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e3e7b9-0a37-4568-9838-cf45fa291f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:04.490168 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e3e7b9-0a37-4568-9838-cf45fa291f31 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:04.491047 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e3e7b9-0a37-4568-9838-cf45fa291f31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:04.493850 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e3e7b9-0a37-4568-9838-cf45fa291f31 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:04.614342 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:04.662433 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1038/2026] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:14:04 2022] POST /identity/v3/projects => generated 397 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:14:04.663013 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-075ac002-6b03-47df-9887-ea83a9cd0614 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0d3a6765ea9469ebd3e526f54904803'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:04.663451 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e3e7b9-0a37-4568-9838-cf45fa291f31 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:04.678358 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-075ac002-6b03-47df-9887-ea83a9cd0614 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0d3a6765ea9469ebd3e526f54904803'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:04.687384 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-075ac002-6b03-47df-9887-ea83a9cd0614 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0d3a6765ea9469ebd3e526f54904803'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:04.688444 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-075ac002-6b03-47df-9887-ea83a9cd0614 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0d3a6765ea9469ebd3e526f54904803'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:04.736453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e3e7b9-0a37-4568-9838-cf45fa291f31 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:04.748494 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd8c2f-0445-40c7-b784-5c16b9a3044f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:04.748494 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd8c2f-0445-40c7-b784-5c16b9a3044f admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:04.748494 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd8c2f-0445-40c7-b784-5c16b9a3044f admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:04.748494 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd8c2f-0445-40c7-b784-5c16b9a3044f admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:04.749618 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dd8c2f-0445-40c7-b784-5c16b9a3044f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:04.749618 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dd8c2f-0445-40c7-b784-5c16b9a3044f admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:04.768186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-075ac002-6b03-47df-9887-ea83a9cd0614 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0d3a6765ea9469ebd3e526f54904803'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:04.775755 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 989/2027] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:14:04 2022] DELETE /identity/v3/users/b0d3a6765ea9469ebd3e526f54904803 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:04.781330 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc00e2-075a-436f-803c-047152f39b29 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:04.781750 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc00e2-075a-436f-803c-047152f39b29 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:04.782339 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc00e2-075a-436f-803c-047152f39b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:04.804567 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-0bdc00e2-075a-436f-803c-047152f39b29 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:04.805648 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-0bdc00e2-075a-436f-803c-047152f39b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:04.866551 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 990/2028] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:14:04.872143 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1039/2029] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:14:04 2022] POST /identity/v3/users => generated 299 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:14:04.876996 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdc00e2-075a-436f-803c-047152f39b29 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:04.878662 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-0bdc00e2-075a-436f-803c-047152f39b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:04.879107 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd8c2f-0445-40c7-b784-5c16b9a3044f admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:04.881051 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c7dd8c2f-0445-40c7-b784-5c16b9a3044f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:04.893828 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdc00e2-075a-436f-803c-047152f39b29 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:04.899390 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d880a50-d2bf-418c-875f-afd631bcbc09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:04.900466 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d880a50-d2bf-418c-875f-afd631bcbc09 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:04.900466 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d880a50-d2bf-418c-875f-afd631bcbc09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:04.900876 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d880a50-d2bf-418c-875f-afd631bcbc09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:04.901944 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d880a50-d2bf-418c-875f-afd631bcbc09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:04.905265 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d880a50-d2bf-418c-875f-afd631bcbc09 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:04.917589 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 991/2030] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:14:04.922399 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d880a50-d2bf-418c-875f-afd631bcbc09 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:04.933184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d880a50-d2bf-418c-875f-afd631bcbc09 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:04.938459 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b7d53a-ebca-4511-976c-b3251c4ca265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:04.939175 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b7d53a-ebca-4511-976c-b3251c4ca265 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:04.939463 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b7d53a-ebca-4511-976c-b3251c4ca265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:04.939758 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b7d53a-ebca-4511-976c-b3251c4ca265 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:04.940540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b7d53a-ebca-4511-976c-b3251c4ca265 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:04.942156 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b7d53a-ebca-4511-976c-b3251c4ca265 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:04.953093 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 992/2031] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:14:04.957548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b7d53a-ebca-4511-976c-b3251c4ca265 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:04.959127 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd8c2f-0445-40c7-b784-5c16b9a3044f admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:04.968008 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b7d53a-ebca-4511-976c-b3251c4ca265 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:04.973842 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c2c62-f619-41a0-94a8-47a6e5f073a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:04.974859 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c2c62-f619-41a0-94a8-47a6e5f073a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:04.975336 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c2c62-f619-41a0-94a8-47a6e5f073a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:04.975717 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c2c62-f619-41a0-94a8-47a6e5f073a9 admin admin] PATH_INFO: `/v3/projects/c9ee5cf1e0ce461c82b1e37a5a222650/users/0d023aca8ffd4e0a82565cae94cab53b/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:04.997057 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c2c62-f619-41a0-94a8-47a6e5f073a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9ee5cf1e0ce461c82b1e37a5a222650, user_id=0d023aca8ffd4e0a82565cae94cab53b, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:05.000753 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c2c62-f619-41a0-94a8-47a6e5f073a9 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:05.044566 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8c0471-4348-4862-9e32-7696d15c42e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:05.045744 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c0471-4348-4862-9e32-7696d15c42e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:05.046025 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c0471-4348-4862-9e32-7696d15c42e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:05.046307 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8c0471-4348-4862-9e32-7696d15c42e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:05.056562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c0471-4348-4862-9e32-7696d15c42e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:05.060483 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8c0471-4348-4862-9e32-7696d15c42e9 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:05.069064 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1040/2032] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:14:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:14:05.098525 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 993/2033] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:14:04 2022] PUT /identity/v3/projects/c9ee5cf1e0ce461c82b1e37a5a222650/users/0d023aca8ffd4e0a82565cae94cab53b/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:05.101393 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5d015-672a-4ce2-87aa-c90608950047 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:05.106360 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5d015-672a-4ce2-87aa-c90608950047 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:05.106360 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5d015-672a-4ce2-87aa-c90608950047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:05.148705 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-98e5d015-672a-4ce2-87aa-c90608950047 None None] MFA Rules not processed for user `0d023aca8ffd4e0a82565cae94cab53b`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:05.153768 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:05.155956 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-98e5d015-672a-4ce2-87aa-c90608950047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:05.288402 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1041/2034] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:05 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:05.299604 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c2c62-f619-41a0-94a8-47a6e5f073a9 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:05.301518 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-900c2c62-f619-41a0-94a8-47a6e5f073a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:05.309266 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:05.374026 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c2c62-f619-41a0-94a8-47a6e5f073a9 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:05.378292 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98313a6-3ba1-4ced-a12d-9f4109cff5e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:05.380031 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98313a6-3ba1-4ced-a12d-9f4109cff5e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:05.380327 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98313a6-3ba1-4ced-a12d-9f4109cff5e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:05.380608 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98313a6-3ba1-4ced-a12d-9f4109cff5e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:05.390811 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98313a6-3ba1-4ced-a12d-9f4109cff5e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:05.392463 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98313a6-3ba1-4ced-a12d-9f4109cff5e9 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:05.404786 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 994/2035] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:14:05 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:14:05.683038 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5d015-672a-4ce2-87aa-c90608950047 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:05.685651 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-98e5d015-672a-4ce2-87aa-c90608950047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:05.795070 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5d015-672a-4ce2-87aa-c90608950047 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:05.799746 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48026471-9cbe-424a-a393-afbc7400f364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:05.800674 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48026471-9cbe-424a-a393-afbc7400f364 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:05.801125 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48026471-9cbe-424a-a393-afbc7400f364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:05.801556 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48026471-9cbe-424a-a393-afbc7400f364 admin admin] PATH_INFO: `/v3/projects/90cf8824908344a687ee5828e257da3c` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:05.802769 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48026471-9cbe-424a-a393-afbc7400f364 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90cf8824908344a687ee5828e257da3c)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:05.804748 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48026471-9cbe-424a-a393-afbc7400f364 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:05.919665 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-48026471-9cbe-424a-a393-afbc7400f364 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90cf8824908344a687ee5828e257da3c', 'request_id': 'req-48026471-9cbe-424a-a393-afbc7400f364'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:05.925966 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-48026471-9cbe-424a-a393-afbc7400f364 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90cf8824908344a687ee5828e257da3c', 'request_id': 'req-48026471-9cbe-424a-a393-afbc7400f364'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:05.931426 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-48026471-9cbe-424a-a393-afbc7400f364 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90cf8824908344a687ee5828e257da3c', 'request_id': 'req-48026471-9cbe-424a-a393-afbc7400f364'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:06.019806 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-48026471-9cbe-424a-a393-afbc7400f364 admin admin] The token cache is being invalidate because project 90cf8824908344a687ee5828e257da3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:14:06.020104 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-48026471-9cbe-424a-a393-afbc7400f364 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:06.020736 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-48026471-9cbe-424a-a393-afbc7400f364 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:06.022317 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1042/2036] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:14:05 2022] DELETE /identity/v3/projects/90cf8824908344a687ee5828e257da3c => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:06.026273 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98313a6-3ba1-4ced-a12d-9f4109cff5e9 service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:06.037297 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98313a6-3ba1-4ced-a12d-9f4109cff5e9 service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:06.102584 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94653ab4-3e61-45e7-8ef2-b223028c72db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:06.103466 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94653ab4-3e61-45e7-8ef2-b223028c72db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:06.103772 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94653ab4-3e61-45e7-8ef2-b223028c72db admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:06.104054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94653ab4-3e61-45e7-8ef2-b223028c72db admin admin] PATH_INFO: `/v3/users/284df019f75544c0ad5ab55233e557ef` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:06.105307 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94653ab4-3e61-45e7-8ef2-b223028c72db admin admin] RBAC: Authorizing `identity:delete_user(user_id=284df019f75544c0ad5ab55233e557ef)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:06.107838 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94653ab4-3e61-45e7-8ef2-b223028c72db admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:06.137170 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:06.305767 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-94653ab4-3e61-45e7-8ef2-b223028c72db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '284df019f75544c0ad5ab55233e557ef'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:06.312240 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:06.317631 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-94653ab4-3e61-45e7-8ef2-b223028c72db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '284df019f75544c0ad5ab55233e557ef'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:06.324307 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-94653ab4-3e61-45e7-8ef2-b223028c72db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '284df019f75544c0ad5ab55233e557ef'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:06.325009 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-94653ab4-3e61-45e7-8ef2-b223028c72db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '284df019f75544c0ad5ab55233e557ef'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:06.397091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-94653ab4-3e61-45e7-8ef2-b223028c72db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '284df019f75544c0ad5ab55233e557ef'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:06.410473 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 995/2037] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:14:06 2022] DELETE /identity/v3/users/284df019f75544c0ad5ab55233e557ef => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:06.847298 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48026471-9cbe-424a-a393-afbc7400f364 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:06.849004 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-48026471-9cbe-424a-a393-afbc7400f364 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:06.852180 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:06.867293 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48026471-9cbe-424a-a393-afbc7400f364 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:06.952577 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0139c-a302-4b74-8100-12c12d21f026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:06.953608 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0139c-a302-4b74-8100-12c12d21f026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:06.953901 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0139c-a302-4b74-8100-12c12d21f026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:06.954173 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0139c-a302-4b74-8100-12c12d21f026 admin admin] PATH_INFO: `/v3/projects/d669d36a808f47deab3ba1c36267b05e` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:06.955452 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c0139c-a302-4b74-8100-12c12d21f026 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d669d36a808f47deab3ba1c36267b05e)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:06.957440 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c0139c-a302-4b74-8100-12c12d21f026 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:07.042194 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-54c0139c-a302-4b74-8100-12c12d21f026 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd669d36a808f47deab3ba1c36267b05e', 'request_id': 'req-54c0139c-a302-4b74-8100-12c12d21f026'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:07.048581 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-54c0139c-a302-4b74-8100-12c12d21f026 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd669d36a808f47deab3ba1c36267b05e', 'request_id': 'req-54c0139c-a302-4b74-8100-12c12d21f026'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:07.054046 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-54c0139c-a302-4b74-8100-12c12d21f026 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd669d36a808f47deab3ba1c36267b05e', 'request_id': 'req-54c0139c-a302-4b74-8100-12c12d21f026'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:07.121041 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-54c0139c-a302-4b74-8100-12c12d21f026 admin admin] The token cache is being invalidate because project d669d36a808f47deab3ba1c36267b05e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:14:07.121339 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-54c0139c-a302-4b74-8100-12c12d21f026 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:07.121984 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-54c0139c-a302-4b74-8100-12c12d21f026 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:07.123573 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1043/2038] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:14:06 2022] DELETE /identity/v3/projects/d669d36a808f47deab3ba1c36267b05e => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:07.132708 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94653ab4-3e61-45e7-8ef2-b223028c72db admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:07.134672 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-94653ab4-3e61-45e7-8ef2-b223028c72db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:07.147991 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94653ab4-3e61-45e7-8ef2-b223028c72db admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:07.197201 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:07.209961 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08054b2-1083-4a7d-b904-0111c0a63fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:07.210884 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08054b2-1083-4a7d-b904-0111c0a63fe8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:07.211167 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08054b2-1083-4a7d-b904-0111c0a63fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:07.211442 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08054b2-1083-4a7d-b904-0111c0a63fe8 admin admin] PATH_INFO: `/v3/users/af120e7625954132802a88898f48233a` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:07.212928 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08054b2-1083-4a7d-b904-0111c0a63fe8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af120e7625954132802a88898f48233a)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:07.214963 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08054b2-1083-4a7d-b904-0111c0a63fe8 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:07.441672 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-b08054b2-1083-4a7d-b904-0111c0a63fe8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af120e7625954132802a88898f48233a'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:07.454054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-b08054b2-1083-4a7d-b904-0111c0a63fe8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af120e7625954132802a88898f48233a'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:07.460843 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-b08054b2-1083-4a7d-b904-0111c0a63fe8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af120e7625954132802a88898f48233a'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:07.461567 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-b08054b2-1083-4a7d-b904-0111c0a63fe8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af120e7625954132802a88898f48233a'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:07.522247 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-b08054b2-1083-4a7d-b904-0111c0a63fe8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af120e7625954132802a88898f48233a'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:07.529071 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 996/2039] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:14:07 2022] DELETE /identity/v3/users/af120e7625954132802a88898f48233a => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:07.602775 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:07.745156 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:08.041900 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0139c-a302-4b74-8100-12c12d21f026 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:08.043397 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-54c0139c-a302-4b74-8100-12c12d21f026 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:08.056829 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0139c-a302-4b74-8100-12c12d21f026 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:08.077410 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:08.137240 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c0b37e-04e3-4b69-8f0d-12d4a183be79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:08.138342 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0b37e-04e3-4b69-8f0d-12d4a183be79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:08.138815 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0b37e-04e3-4b69-8f0d-12d4a183be79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:08.139148 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c0b37e-04e3-4b69-8f0d-12d4a183be79 admin admin] PATH_INFO: `/v3/projects/2bffae38482b47519d492494d213aaab` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:08.140555 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c0b37e-04e3-4b69-8f0d-12d4a183be79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bffae38482b47519d492494d213aaab)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:08.142511 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c0b37e-04e3-4b69-8f0d-12d4a183be79 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:08.157359 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:08.230895 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=5,keystone:INSERT=6 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:08.278036 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-85c0b37e-04e3-4b69-8f0d-12d4a183be79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bffae38482b47519d492494d213aaab', 'request_id': 'req-85c0b37e-04e3-4b69-8f0d-12d4a183be79'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:08.285231 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-85c0b37e-04e3-4b69-8f0d-12d4a183be79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bffae38482b47519d492494d213aaab', 'request_id': 'req-85c0b37e-04e3-4b69-8f0d-12d4a183be79'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:08.291444 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-85c0b37e-04e3-4b69-8f0d-12d4a183be79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bffae38482b47519d492494d213aaab', 'request_id': 'req-85c0b37e-04e3-4b69-8f0d-12d4a183be79'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:08.356379 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-85c0b37e-04e3-4b69-8f0d-12d4a183be79 admin admin] The token cache is being invalidate because project 2bffae38482b47519d492494d213aaab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:14:08.356703 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-85c0b37e-04e3-4b69-8f0d-12d4a183be79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:08.357400 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-85c0b37e-04e3-4b69-8f0d-12d4a183be79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:08.359124 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1044/2040] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:14:08 2022] DELETE /identity/v3/projects/2bffae38482b47519d492494d213aaab => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:08.367995 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784124af-cc1d-4305-b36a-7a7c1642ee69 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:08.368286 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784124af-cc1d-4305-b36a-7a7c1642ee69 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:08.368900 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784124af-cc1d-4305-b36a-7a7c1642ee69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:08.392557 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-784124af-cc1d-4305-b36a-7a7c1642ee69 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:08.394070 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-784124af-cc1d-4305-b36a-7a7c1642ee69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:08.436809 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:08.457339 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 997/2041] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:08.465148 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8d9ef-366f-4a67-a2c7-392a35ad6da3 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:08.465452 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8d9ef-366f-4a67-a2c7-392a35ad6da3 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:08.465758 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e8d9ef-366f-4a67-a2c7-392a35ad6da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:08.484751 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-76e8d9ef-366f-4a67-a2c7-392a35ad6da3 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:08.485520 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-76e8d9ef-366f-4a67-a2c7-392a35ad6da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:08.493629 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1045/2042] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:08.498541 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784124af-cc1d-4305-b36a-7a7c1642ee69 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:08.508256 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784124af-cc1d-4305-b36a-7a7c1642ee69 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:08.512888 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a953aa-9db7-441a-8a0d-383d608664f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:08.514377 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a953aa-9db7-441a-8a0d-383d608664f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:08.514670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a953aa-9db7-441a-8a0d-383d608664f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:08.514953 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a953aa-9db7-441a-8a0d-383d608664f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:08.515695 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a953aa-9db7-441a-8a0d-383d608664f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:14:08.516045 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a953aa-9db7-441a-8a0d-383d608664f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:08.517656 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a953aa-9db7-441a-8a0d-383d608664f8 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:08.530250 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 998/2043] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:14:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:14:08.534933 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e8d9ef-366f-4a67-a2c7-392a35ad6da3 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:08.544165 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e8d9ef-366f-4a67-a2c7-392a35ad6da3 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:08.548929 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b7e4d-3dbb-4623-a4eb-dfe34d913ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:08.549567 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b7e4d-3dbb-4623-a4eb-dfe34d913ea4 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:08.549851 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b7e4d-3dbb-4623-a4eb-dfe34d913ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:08.550125 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237b7e4d-3dbb-4623-a4eb-dfe34d913ea4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:08.550821 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b7e4d-3dbb-4623-a4eb-dfe34d913ea4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:08.552947 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237b7e4d-3dbb-4623-a4eb-dfe34d913ea4 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:08.649488 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1046/2044] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:14:08 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:14:08.654703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a953aa-9db7-441a-8a0d-383d608664f8 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:08.654826 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:08.664819 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a953aa-9db7-441a-8a0d-383d608664f8 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:08.723157 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba5ccfb-1e9b-493a-abb7-bf67ee838280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:08.723870 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba5ccfb-1e9b-493a-abb7-bf67ee838280 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:08.724160 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba5ccfb-1e9b-493a-abb7-bf67ee838280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:08.724436 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba5ccfb-1e9b-493a-abb7-bf67ee838280 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:08.725123 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba5ccfb-1e9b-493a-abb7-bf67ee838280 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:08.727101 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba5ccfb-1e9b-493a-abb7-bf67ee838280 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:08.731272 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:08.819840 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 999/2045] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:14:08 2022] POST /identity/v3/users => generated 287 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:14:08.828457 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b7e4d-3dbb-4623-a4eb-dfe34d913ea4 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:08.838617 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237b7e4d-3dbb-4623-a4eb-dfe34d913ea4 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:08.843271 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5fa8eb-baa9-4e5d-8e05-cf1a2eff6700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:08.843968 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5fa8eb-baa9-4e5d-8e05-cf1a2eff6700 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:08.844252 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5fa8eb-baa9-4e5d-8e05-cf1a2eff6700 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:08.844546 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5fa8eb-baa9-4e5d-8e05-cf1a2eff6700 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:08.845245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5fa8eb-baa9-4e5d-8e05-cf1a2eff6700 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:08.846979 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5fa8eb-baa9-4e5d-8e05-cf1a2eff6700 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:08.857297 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1047/2046] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:14:08.861201 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba5ccfb-1e9b-493a-abb7-bf67ee838280 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:08.871055 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba5ccfb-1e9b-493a-abb7-bf67ee838280 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:08.876378 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf83aad-8bea-4e5e-8933-cad12e041b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:08.877039 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf83aad-8bea-4e5e-8933-cad12e041b0c admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:08.880083 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf83aad-8bea-4e5e-8933-cad12e041b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:08.880083 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf83aad-8bea-4e5e-8933-cad12e041b0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:08.880083 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf83aad-8bea-4e5e-8933-cad12e041b0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:08.880083 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf83aad-8bea-4e5e-8933-cad12e041b0c admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:08.890454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1000/2047] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:14:08.894545 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5fa8eb-baa9-4e5d-8e05-cf1a2eff6700 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:08.899535 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:08.904676 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5fa8eb-baa9-4e5d-8e05-cf1a2eff6700 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:08.909243 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0935d9f9-8e94-4cdc-807e-1839fd4f7df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:08.910111 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0935d9f9-8e94-4cdc-807e-1839fd4f7df0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:08.910396 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0935d9f9-8e94-4cdc-807e-1839fd4f7df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:08.910670 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0935d9f9-8e94-4cdc-807e-1839fd4f7df0 admin admin] PATH_INFO: `/v3/projects/9f85d0f3814648d18a627e1353493973/users/2601402e569e491a94cea7a312cfef86/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:08.924862 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0935d9f9-8e94-4cdc-807e-1839fd4f7df0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f85d0f3814648d18a627e1353493973, user_id=2601402e569e491a94cea7a312cfef86, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:08.927534 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0935d9f9-8e94-4cdc-807e-1839fd4f7df0 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:08.957261 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:08.985282 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1048/2048] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:14:08 2022] PUT /identity/v3/projects/9f85d0f3814648d18a627e1353493973/users/2601402e569e491a94cea7a312cfef86/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:08.990396 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be0cdd-5a10-4ce4-be62-f8d601abd852 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:08.990639 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be0cdd-5a10-4ce4-be62-f8d601abd852 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:08.990957 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28be0cdd-5a10-4ce4-be62-f8d601abd852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:09.008296 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:09.008886 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-28be0cdd-5a10-4ce4-be62-f8d601abd852 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:09.010085 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-28be0cdd-5a10-4ce4-be62-f8d601abd852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:09.068447 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1001/2049] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:09.077825 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0935d9f9-8e94-4cdc-807e-1839fd4f7df0 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:09.079601 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-0935d9f9-8e94-4cdc-807e-1839fd4f7df0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:09.136906 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0935d9f9-8e94-4cdc-807e-1839fd4f7df0 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:09.140671 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948add8-6fc6-4b24-acd1-89a753e6bf63 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:09.141320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948add8-6fc6-4b24-acd1-89a753e6bf63 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:09.141609 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948add8-6fc6-4b24-acd1-89a753e6bf63 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:09.141889 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948add8-6fc6-4b24-acd1-89a753e6bf63 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:09.145409 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=81,keystone:INSERT=3,keystone:DELETE=16 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:09.150905 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a948add8-6fc6-4b24-acd1-89a753e6bf63 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:09.155494 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a948add8-6fc6-4b24-acd1-89a753e6bf63 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:09.162232 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1049/2050] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:14:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:14:09.174766 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:09.522517 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:09.568499 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:09.808429 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:10.018889 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:10.117271 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:10.214502 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:10.347302 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:10.391523 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:10.769375 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:10.806941 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:11.218263 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:11.371607 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:11.855602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:11.962772 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:12.493183 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:12.541313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:13.208405 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:13.332489 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28be0cdd-5a10-4ce4-be62-f8d601abd852 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:13.334050 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-28be0cdd-5a10-4ce4-be62-f8d601abd852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:13.364352 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28be0cdd-5a10-4ce4-be62-f8d601abd852 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:13.370062 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77008602-d952-4f70-a39b-da19c6555b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:13.371172 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77008602-d952-4f70-a39b-da19c6555b0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:13.371641 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77008602-d952-4f70-a39b-da19c6555b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:13.372233 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77008602-d952-4f70-a39b-da19c6555b0a admin admin] PATH_INFO: `/v3/users/452219fec55e4c6abf58e7fa4d91ac5f` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:13.373715 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77008602-d952-4f70-a39b-da19c6555b0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=452219fec55e4c6abf58e7fa4d91ac5f)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:13.375940 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77008602-d952-4f70-a39b-da19c6555b0a admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:13.580777 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-77008602-d952-4f70-a39b-da19c6555b0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '452219fec55e4c6abf58e7fa4d91ac5f'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:13.599399 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-77008602-d952-4f70-a39b-da19c6555b0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '452219fec55e4c6abf58e7fa4d91ac5f'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:13.607943 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-77008602-d952-4f70-a39b-da19c6555b0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '452219fec55e4c6abf58e7fa4d91ac5f'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:13.608692 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-77008602-d952-4f70-a39b-da19c6555b0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '452219fec55e4c6abf58e7fa4d91ac5f'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:13.682475 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-77008602-d952-4f70-a39b-da19c6555b0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '452219fec55e4c6abf58e7fa4d91ac5f'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:13.688853 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1002/2051] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:14:13 2022] DELETE /identity/v3/users/452219fec55e4c6abf58e7fa4d91ac5f => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:13.693349 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979acbb-de88-4abf-abaf-254e584db176 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:13.693819 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979acbb-de88-4abf-abaf-254e584db176 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:13.694072 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979acbb-de88-4abf-abaf-254e584db176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:13.732907 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-5979acbb-de88-4abf-abaf-254e584db176 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:13.738663 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-5979acbb-de88-4abf-abaf-254e584db176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:13.816588 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1050/2052] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:14:13.826903 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77008602-d952-4f70-a39b-da19c6555b0a admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:13.828659 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-77008602-d952-4f70-a39b-da19c6555b0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:13.889347 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77008602-d952-4f70-a39b-da19c6555b0a admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:13.892982 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c11c2-d661-452c-9c21-bca103d40c07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:13.898709 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c11c2-d661-452c-9c21-bca103d40c07 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:13.898709 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c11c2-d661-452c-9c21-bca103d40c07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:13.898709 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c11c2-d661-452c-9c21-bca103d40c07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:13.904190 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4c11c2-d661-452c-9c21-bca103d40c07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:13.905630 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4c11c2-d661-452c-9c21-bca103d40c07 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:13.915126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1003/2053] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:14:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:14:13.997767 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:14.258837 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:14.450902 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5979acbb-de88-4abf-abaf-254e584db176 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:14.453219 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-5979acbb-de88-4abf-abaf-254e584db176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:14.470819 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5979acbb-de88-4abf-abaf-254e584db176 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:14.484755 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5b0a2b-bc5c-4ee4-a59f-e9be911c0fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:14.485692 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b0a2b-bc5c-4ee4-a59f-e9be911c0fac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:14.485976 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b0a2b-bc5c-4ee4-a59f-e9be911c0fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:14.486570 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5b0a2b-bc5c-4ee4-a59f-e9be911c0fac admin admin] PATH_INFO: `/v3/projects/bbb4139cfb67483b935200ba0d9d3ba0` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:14.488039 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5b0a2b-bc5c-4ee4-a59f-e9be911c0fac admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbb4139cfb67483b935200ba0d9d3ba0)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:14.490059 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5b0a2b-bc5c-4ee4-a59f-e9be911c0fac admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:14.628004 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0f5b0a2b-bc5c-4ee4-a59f-e9be911c0fac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbb4139cfb67483b935200ba0d9d3ba0', 'request_id': 'req-0f5b0a2b-bc5c-4ee4-a59f-e9be911c0fac'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:14.647256 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0f5b0a2b-bc5c-4ee4-a59f-e9be911c0fac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbb4139cfb67483b935200ba0d9d3ba0', 'request_id': 'req-0f5b0a2b-bc5c-4ee4-a59f-e9be911c0fac'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:14.657858 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0f5b0a2b-bc5c-4ee4-a59f-e9be911c0fac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbb4139cfb67483b935200ba0d9d3ba0', 'request_id': 'req-0f5b0a2b-bc5c-4ee4-a59f-e9be911c0fac'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:14.741789 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:14.763483 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0f5b0a2b-bc5c-4ee4-a59f-e9be911c0fac admin admin] The token cache is being invalidate because project bbb4139cfb67483b935200ba0d9d3ba0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:14:14.763833 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0f5b0a2b-bc5c-4ee4-a59f-e9be911c0fac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:14.768228 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0f5b0a2b-bc5c-4ee4-a59f-e9be911c0fac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:14.772176 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1051/2054] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:14:14 2022] DELETE /identity/v3/projects/bbb4139cfb67483b935200ba0d9d3ba0 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:14.774744 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c11c2-d661-452c-9c21-bca103d40c07 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:14.787406 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4c11c2-d661-452c-9c21-bca103d40c07 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:14.888805 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10203899-0cf6-487b-a43f-19dd78b78397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:14.889840 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10203899-0cf6-487b-a43f-19dd78b78397 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:14.890191 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10203899-0cf6-487b-a43f-19dd78b78397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:14.890537 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10203899-0cf6-487b-a43f-19dd78b78397 admin admin] PATH_INFO: `/v3/users/da89e0b6c89e4c0bb7008c69d7d74949` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:14.892008 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10203899-0cf6-487b-a43f-19dd78b78397 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da89e0b6c89e4c0bb7008c69d7d74949)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:14.894270 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10203899-0cf6-487b-a43f-19dd78b78397 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:14.946242 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:15.080384 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d1079-b877-456c-9d14-78d0141d4ad2 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:15.080674 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d1079-b877-456c-9d14-78d0141d4ad2 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:15.081163 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3d1079-b877-456c-9d14-78d0141d4ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:15.081282 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-10203899-0cf6-487b-a43f-19dd78b78397 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da89e0b6c89e4c0bb7008c69d7d74949'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:15.100552 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-10203899-0cf6-487b-a43f-19dd78b78397 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da89e0b6c89e4c0bb7008c69d7d74949'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:15.112378 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-ba3d1079-b877-456c-9d14-78d0141d4ad2 None None] MFA Rules not processed for user `2601402e569e491a94cea7a312cfef86`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:15.113611 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-ba3d1079-b877-456c-9d14-78d0141d4ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:15.113733 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-10203899-0cf6-487b-a43f-19dd78b78397 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da89e0b6c89e4c0bb7008c69d7d74949'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:15.114364 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-10203899-0cf6-487b-a43f-19dd78b78397 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da89e0b6c89e4c0bb7008c69d7d74949'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:15.173572 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-10203899-0cf6-487b-a43f-19dd78b78397 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da89e0b6c89e4c0bb7008c69d7d74949'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:15.180824 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1004/2055] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:14:14 2022] DELETE /identity/v3/users/da89e0b6c89e4c0bb7008c69d7d74949 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:15.193657 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1052/2056] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:14:15.199133 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10203899-0cf6-487b-a43f-19dd78b78397 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:15.201114 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-10203899-0cf6-487b-a43f-19dd78b78397 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:15.273508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10203899-0cf6-487b-a43f-19dd78b78397 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:15.279681 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6bdb4b-6581-4075-a5f6-a35137ef1342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:15.280388 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6bdb4b-6581-4075-a5f6-a35137ef1342 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:15.281169 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6bdb4b-6581-4075-a5f6-a35137ef1342 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:15.281460 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6bdb4b-6581-4075-a5f6-a35137ef1342 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:15.282171 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6bdb4b-6581-4075-a5f6-a35137ef1342 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:15.284526 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6bdb4b-6581-4075-a5f6-a35137ef1342 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:15.365206 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1005/2057] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:14:15 2022] POST /identity/v3/projects => generated 371 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:14:15.369199 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d1079-b877-456c-9d14-78d0141d4ad2 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:15.371501 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-ba3d1079-b877-456c-9d14-78d0141d4ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:15.389121 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3d1079-b877-456c-9d14-78d0141d4ad2 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:15.481404 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced609d-0f6c-40b2-84e0-a11d57a72694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:15.482157 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced609d-0f6c-40b2-84e0-a11d57a72694 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:15.482443 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced609d-0f6c-40b2-84e0-a11d57a72694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:15.482807 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ced609d-0f6c-40b2-84e0-a11d57a72694 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:15.483524 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ced609d-0f6c-40b2-84e0-a11d57a72694 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:15.485681 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ced609d-0f6c-40b2-84e0-a11d57a72694 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:15.559441 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1053/2058] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:14:15 2022] POST /identity/v3/users => generated 292 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:14:15.561492 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6bdb4b-6581-4075-a5f6-a35137ef1342 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:15.563847 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:15.573671 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6bdb4b-6581-4075-a5f6-a35137ef1342 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:15.578951 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a01cb95-9944-4026-a8bc-3f97adc415a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:15.579614 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01cb95-9944-4026-a8bc-3f97adc415a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:15.579998 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01cb95-9944-4026-a8bc-3f97adc415a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:15.580280 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01cb95-9944-4026-a8bc-3f97adc415a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:15.581012 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a01cb95-9944-4026-a8bc-3f97adc415a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:15.582617 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a01cb95-9944-4026-a8bc-3f97adc415a7 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:15.593459 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1006/2059] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:14:15.597708 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced609d-0f6c-40b2-84e0-a11d57a72694 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:15.608041 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ced609d-0f6c-40b2-84e0-a11d57a72694 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:15.612938 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73068c-6cfe-48ae-8c58-9cf548a2a601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:15.613825 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73068c-6cfe-48ae-8c58-9cf548a2a601 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:15.614108 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73068c-6cfe-48ae-8c58-9cf548a2a601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:15.614380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab73068c-6cfe-48ae-8c58-9cf548a2a601 admin admin] PATH_INFO: `/v3/projects/9f3fa70ae6ea49568bd152e70b5f7b86/users/eb0e1e84f287480880a3150164e5a467/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:15.631277 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab73068c-6cfe-48ae-8c58-9cf548a2a601 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f3fa70ae6ea49568bd152e70b5f7b86, user_id=eb0e1e84f287480880a3150164e5a467, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:15.635432 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab73068c-6cfe-48ae-8c58-9cf548a2a601 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:15.680414 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a01cb95-9944-4026-a8bc-3f97adc415a7 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:15.682184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4a01cb95-9944-4026-a8bc-3f97adc415a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:15.694236 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:15.695975 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a01cb95-9944-4026-a8bc-3f97adc415a7 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:15.701310 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a530a6f-a020-4b94-bfe5-5d47c88ea414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:15.703188 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a530a6f-a020-4b94-bfe5-5d47c88ea414 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:15.703310 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a530a6f-a020-4b94-bfe5-5d47c88ea414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:15.703598 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a530a6f-a020-4b94-bfe5-5d47c88ea414 admin admin] PATH_INFO: `/v3/projects/134c4d05e49045adb315afaaea428025` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:15.704900 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a530a6f-a020-4b94-bfe5-5d47c88ea414 admin admin] RBAC: Authorizing `identity:delete_project(project_id=134c4d05e49045adb315afaaea428025)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:15.706858 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a530a6f-a020-4b94-bfe5-5d47c88ea414 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:15.745684 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1054/2060] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:14:15 2022] PUT /identity/v3/projects/9f3fa70ae6ea49568bd152e70b5f7b86/users/eb0e1e84f287480880a3150164e5a467/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:15.752436 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73068c-6cfe-48ae-8c58-9cf548a2a601 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:15.796611 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:15.828430 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab73068c-6cfe-48ae-8c58-9cf548a2a601 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:15.837559 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a049fd5-12bc-45cb-8455-7bc9a4797c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:15.837559 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a049fd5-12bc-45cb-8455-7bc9a4797c18 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:15.837559 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a049fd5-12bc-45cb-8455-7bc9a4797c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:15.837559 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a049fd5-12bc-45cb-8455-7bc9a4797c18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:15.838263 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a049fd5-12bc-45cb-8455-7bc9a4797c18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:15.838263 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a049fd5-12bc-45cb-8455-7bc9a4797c18 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:15.851115 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1055/2061] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:15 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:14:15.860489 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a049fd5-12bc-45cb-8455-7bc9a4797c18 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:15.870373 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-9a530a6f-a020-4b94-bfe5-5d47c88ea414 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '134c4d05e49045adb315afaaea428025', 'request_id': 'req-9a530a6f-a020-4b94-bfe5-5d47c88ea414'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:15.873332 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a049fd5-12bc-45cb-8455-7bc9a4797c18 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:15.887186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-9a530a6f-a020-4b94-bfe5-5d47c88ea414 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '134c4d05e49045adb315afaaea428025', 'request_id': 'req-9a530a6f-a020-4b94-bfe5-5d47c88ea414'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:15.963420 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95ef6f1-fffb-4ddd-bfdb-22980d485e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:15.963420 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ef6f1-fffb-4ddd-bfdb-22980d485e47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:15.963420 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ef6f1-fffb-4ddd-bfdb-22980d485e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:15.963420 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95ef6f1-fffb-4ddd-bfdb-22980d485e47 admin admin] PATH_INFO: `/v3/projects/9f3fa70ae6ea49568bd152e70b5f7b86/users/eb0e1e84f287480880a3150164e5a467/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:15.964020 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ef6f1-fffb-4ddd-bfdb-22980d485e47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f3fa70ae6ea49568bd152e70b5f7b86, user_id=eb0e1e84f287480880a3150164e5a467, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:15.964020 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95ef6f1-fffb-4ddd-bfdb-22980d485e47 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:15.973169 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1056/2062] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:14:15 2022] PUT /identity/v3/projects/9f3fa70ae6ea49568bd152e70b5f7b86/users/eb0e1e84f287480880a3150164e5a467/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:16.077012 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-9a530a6f-a020-4b94-bfe5-5d47c88ea414 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '134c4d05e49045adb315afaaea428025', 'request_id': 'req-9a530a6f-a020-4b94-bfe5-5d47c88ea414'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:16.094574 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-9a530a6f-a020-4b94-bfe5-5d47c88ea414 admin admin] The token cache is being invalidate because project 134c4d05e49045adb315afaaea428025 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:14:16.094948 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-9a530a6f-a020-4b94-bfe5-5d47c88ea414 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:16.095956 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-9a530a6f-a020-4b94-bfe5-5d47c88ea414 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:16.097766 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1007/2063] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:14:15 2022] DELETE /identity/v3/projects/134c4d05e49045adb315afaaea428025 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:16.102874 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493ae63-d333-44c1-8737-8c623b019b0b None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:16.103172 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493ae63-d333-44c1-8737-8c623b019b0b None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:16.103593 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c493ae63-d333-44c1-8737-8c623b019b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:16.123483 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-c493ae63-d333-44c1-8737-8c623b019b0b None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:16.124976 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c493ae63-d333-44c1-8737-8c623b019b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:16.211359 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1057/2064] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:16.219750 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896ce73-d7ae-4a5c-afe9-6d08ae2f53dd None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:16.220022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896ce73-d7ae-4a5c-afe9-6d08ae2f53dd None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:16.220288 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896ce73-d7ae-4a5c-afe9-6d08ae2f53dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:16.239411 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-a896ce73-d7ae-4a5c-afe9-6d08ae2f53dd None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:16.240641 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-a896ce73-d7ae-4a5c-afe9-6d08ae2f53dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:16.250332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1008/2065] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:16.261741 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c493ae63-d333-44c1-8737-8c623b019b0b None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:16.273687 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c493ae63-d333-44c1-8737-8c623b019b0b None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:16.279624 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72085a2c-46a3-4028-b7ac-c39e86cbbe0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:16.280350 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72085a2c-46a3-4028-b7ac-c39e86cbbe0f admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:16.280625 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72085a2c-46a3-4028-b7ac-c39e86cbbe0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:16.280902 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72085a2c-46a3-4028-b7ac-c39e86cbbe0f admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:16.281738 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72085a2c-46a3-4028-b7ac-c39e86cbbe0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:14:16.282046 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72085a2c-46a3-4028-b7ac-c39e86cbbe0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:16.283791 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72085a2c-46a3-4028-b7ac-c39e86cbbe0f admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:16.296002 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1058/2066] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:14:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:14:16.300584 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a896ce73-d7ae-4a5c-afe9-6d08ae2f53dd None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:16.313473 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a896ce73-d7ae-4a5c-afe9-6d08ae2f53dd None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:16.321327 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979bc4c8-87a7-4f8f-a5f9-264a5334c25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:16.321327 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979bc4c8-87a7-4f8f-a5f9-264a5334c25e admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:16.321327 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979bc4c8-87a7-4f8f-a5f9-264a5334c25e admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:16.321327 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979bc4c8-87a7-4f8f-a5f9-264a5334c25e admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:16.321677 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979bc4c8-87a7-4f8f-a5f9-264a5334c25e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:16.322625 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979bc4c8-87a7-4f8f-a5f9-264a5334c25e admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:16.429881 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1009/2067] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:14:16 2022] POST /identity/v3/projects => generated 381 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:14:16.430085 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72085a2c-46a3-4028-b7ac-c39e86cbbe0f admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:16.442153 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72085a2c-46a3-4028-b7ac-c39e86cbbe0f admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:16.557534 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:16.568426 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80876cb9-d714-42b3-b339-af2f33df0029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:16.569120 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80876cb9-d714-42b3-b339-af2f33df0029 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:16.569405 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80876cb9-d714-42b3-b339-af2f33df0029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:16.569681 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80876cb9-d714-42b3-b339-af2f33df0029 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:16.570426 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80876cb9-d714-42b3-b339-af2f33df0029 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:16.572325 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80876cb9-d714-42b3-b339-af2f33df0029 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:16.646059 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1059/2068] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:14:16 2022] POST /identity/v3/users => generated 291 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:14:16.649711 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979bc4c8-87a7-4f8f-a5f9-264a5334c25e admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:16.666989 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979bc4c8-87a7-4f8f-a5f9-264a5334c25e admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:16.674146 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34126ec2-3d1d-46cd-93ce-d0339fbdf751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:16.675144 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34126ec2-3d1d-46cd-93ce-d0339fbdf751 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:16.675722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34126ec2-3d1d-46cd-93ce-d0339fbdf751 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:16.676196 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34126ec2-3d1d-46cd-93ce-d0339fbdf751 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:16.677178 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34126ec2-3d1d-46cd-93ce-d0339fbdf751 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:16.679340 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34126ec2-3d1d-46cd-93ce-d0339fbdf751 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:16.708878 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1010/2069] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:14:16.712499 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80876cb9-d714-42b3-b339-af2f33df0029 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:16.732924 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80876cb9-d714-42b3-b339-af2f33df0029 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:16.739275 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25765506-3dd3-4a15-9e4c-77bcd9095c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:16.740210 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25765506-3dd3-4a15-9e4c-77bcd9095c33 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:16.740501 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25765506-3dd3-4a15-9e4c-77bcd9095c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:16.740781 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25765506-3dd3-4a15-9e4c-77bcd9095c33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:16.741535 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25765506-3dd3-4a15-9e4c-77bcd9095c33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:16.743633 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25765506-3dd3-4a15-9e4c-77bcd9095c33 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:16.764401 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1060/2070] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:16 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:14:16.768646 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34126ec2-3d1d-46cd-93ce-d0339fbdf751 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:16.781430 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34126ec2-3d1d-46cd-93ce-d0339fbdf751 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:16.786458 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d1396d-bc2f-451f-899f-3c1a0f8aa531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:16.787372 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1396d-bc2f-451f-899f-3c1a0f8aa531 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:16.787661 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1396d-bc2f-451f-899f-3c1a0f8aa531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:16.788247 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1396d-bc2f-451f-899f-3c1a0f8aa531 admin admin] PATH_INFO: `/v3/projects/e8c99d58ff2644599d70789319ec850a/users/ff36048f14214b779711e7230b51f426/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:16.806402 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d1396d-bc2f-451f-899f-3c1a0f8aa531 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8c99d58ff2644599d70789319ec850a, user_id=ff36048f14214b779711e7230b51f426, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:16.809312 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d1396d-bc2f-451f-899f-3c1a0f8aa531 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:16.891642 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1011/2071] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:14:16 2022] PUT /identity/v3/projects/e8c99d58ff2644599d70789319ec850a/users/ff36048f14214b779711e7230b51f426/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:16.907820 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68a4c3-0cf8-41bb-a7e6-a235d6506542 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:16.908512 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68a4c3-0cf8-41bb-a7e6-a235d6506542 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:16.908512 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc68a4c3-0cf8-41bb-a7e6-a235d6506542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:16.938442 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-bc68a4c3-0cf8-41bb-a7e6-a235d6506542 None None] MFA Rules not processed for user `ff36048f14214b779711e7230b51f426`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:16.939685 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-bc68a4c3-0cf8-41bb-a7e6-a235d6506542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:16.975054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:17.014585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1061/2072] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:14:17.019231 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d1396d-bc2f-451f-899f-3c1a0f8aa531 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:17.083382 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d1396d-bc2f-451f-899f-3c1a0f8aa531 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:17.088335 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb165f57-4fde-4eff-aa4b-80231b92e430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:17.089104 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb165f57-4fde-4eff-aa4b-80231b92e430 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:17.089503 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb165f57-4fde-4eff-aa4b-80231b92e430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:17.089870 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb165f57-4fde-4eff-aa4b-80231b92e430 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:17.090648 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb165f57-4fde-4eff-aa4b-80231b92e430 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:17.092808 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb165f57-4fde-4eff-aa4b-80231b92e430 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:17.210628 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1012/2073] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:14:17 2022] POST /identity/v3/projects => generated 381 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:14:17.219359 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68a4c3-0cf8-41bb-a7e6-a235d6506542 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:17.231064 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc68a4c3-0cf8-41bb-a7e6-a235d6506542 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:17.302616 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1a104-a381-4ed6-9265-dfceb7a5d484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:17.303425 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1a104-a381-4ed6-9265-dfceb7a5d484 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:17.303715 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1a104-a381-4ed6-9265-dfceb7a5d484 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:17.304030 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c1a104-a381-4ed6-9265-dfceb7a5d484 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:17.304733 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c1a104-a381-4ed6-9265-dfceb7a5d484 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:17.306676 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c1a104-a381-4ed6-9265-dfceb7a5d484 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:17.409893 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1062/2074] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:14:17 2022] POST /identity/v3/users => generated 297 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:14:17.413928 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb165f57-4fde-4eff-aa4b-80231b92e430 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:17.424829 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb165f57-4fde-4eff-aa4b-80231b92e430 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:17.429858 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23952c9f-bdd8-4101-a543-5cbe45fd30ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:17.430699 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23952c9f-bdd8-4101-a543-5cbe45fd30ae admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:17.430919 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23952c9f-bdd8-4101-a543-5cbe45fd30ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:17.431200 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23952c9f-bdd8-4101-a543-5cbe45fd30ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:17.432013 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23952c9f-bdd8-4101-a543-5cbe45fd30ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:17.433636 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23952c9f-bdd8-4101-a543-5cbe45fd30ae admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:17.446212 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1013/2075] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:14:17.450406 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1a104-a381-4ed6-9265-dfceb7a5d484 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:17.461454 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c1a104-a381-4ed6-9265-dfceb7a5d484 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:17.469363 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb66767b-b2b5-4b43-a660-e98c8669ee72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:17.470148 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb66767b-b2b5-4b43-a660-e98c8669ee72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:17.470432 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb66767b-b2b5-4b43-a660-e98c8669ee72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:17.470928 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb66767b-b2b5-4b43-a660-e98c8669ee72 admin admin] PATH_INFO: `/v3/projects/afd9750067ed47bd8d83126b280d5b32/users/d2aec9298c524d59a159144d88fd5486/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:17.494480 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb66767b-b2b5-4b43-a660-e98c8669ee72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afd9750067ed47bd8d83126b280d5b32, user_id=d2aec9298c524d59a159144d88fd5486, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:17.498168 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb66767b-b2b5-4b43-a660-e98c8669ee72 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:17.613192 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1063/2076] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:14:17 2022] PUT /identity/v3/projects/afd9750067ed47bd8d83126b280d5b32/users/d2aec9298c524d59a159144d88fd5486/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:17.616993 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23952c9f-bdd8-4101-a543-5cbe45fd30ae admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:17.627537 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23952c9f-bdd8-4101-a543-5cbe45fd30ae admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:17.665342 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:17.696854 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc3e68-0365-4375-baa5-42e399535aa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:17.697649 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc3e68-0365-4375-baa5-42e399535aa1 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:17.697882 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc3e68-0365-4375-baa5-42e399535aa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:17.698223 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc3e68-0365-4375-baa5-42e399535aa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:17.699229 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc3e68-0365-4375-baa5-42e399535aa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:17.702439 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccc3e68-0365-4375-baa5-42e399535aa1 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:17.718308 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1014/2077] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:17 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:14:17.722417 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb66767b-b2b5-4b43-a660-e98c8669ee72 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:17.733779 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb66767b-b2b5-4b43-a660-e98c8669ee72 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:17.739425 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd25597-d39b-4749-894f-409625cc768c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:17.739881 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd25597-d39b-4749-894f-409625cc768c admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:17.740329 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd25597-d39b-4749-894f-409625cc768c admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:17.740470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd25597-d39b-4749-894f-409625cc768c admin admin] PATH_INFO: `/v3/projects/afd9750067ed47bd8d83126b280d5b32/users/d2aec9298c524d59a159144d88fd5486/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:17.742827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd25597-d39b-4749-894f-409625cc768c admin admin] RBAC: Authorizing `identity:create_grant(project_id=afd9750067ed47bd8d83126b280d5b32, user_id=d2aec9298c524d59a159144d88fd5486, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:17.745479 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd25597-d39b-4749-894f-409625cc768c admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:17.759816 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1064/2078] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:14:17 2022] PUT /identity/v3/projects/afd9750067ed47bd8d83126b280d5b32/users/d2aec9298c524d59a159144d88fd5486/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:17.770920 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dec48a-06c5-480c-87e2-4ddc18fe48bc None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:17.770920 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dec48a-06c5-480c-87e2-4ddc18fe48bc None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:17.771196 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dec48a-06c5-480c-87e2-4ddc18fe48bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:17.806983 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-04dec48a-06c5-480c-87e2-4ddc18fe48bc None None] MFA Rules not processed for user `d2aec9298c524d59a159144d88fd5486`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:17.808264 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-04dec48a-06c5-480c-87e2-4ddc18fe48bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:17.903905 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1015/2079] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:17 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:17.929412 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd25597-d39b-4749-894f-409625cc768c admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:17.931307 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-cbd25597-d39b-4749-894f-409625cc768c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:18.002696 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd25597-d39b-4749-894f-409625cc768c admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:18.008622 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e34ca-3443-48c4-8ad4-44cbcfdc9c8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:18.009399 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e34ca-3443-48c4-8ad4-44cbcfdc9c8c service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:18.009750 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e34ca-3443-48c4-8ad4-44cbcfdc9c8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:18.010209 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0e34ca-3443-48c4-8ad4-44cbcfdc9c8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:18.032710 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0e34ca-3443-48c4-8ad4-44cbcfdc9c8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:18.034824 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0e34ca-3443-48c4-8ad4-44cbcfdc9c8c service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:18.055010 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1065/2080] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:14:17 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:14:18.242964 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dec48a-06c5-480c-87e2-4ddc18fe48bc None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:18.261600 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dec48a-06c5-480c-87e2-4ddc18fe48bc None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:18.268355 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2991aacc-b0b5-4e1f-a3b2-0b3ea516edc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:18.269256 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991aacc-b0b5-4e1f-a3b2-0b3ea516edc8 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:18.269545 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991aacc-b0b5-4e1f-a3b2-0b3ea516edc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:18.269828 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2991aacc-b0b5-4e1f-a3b2-0b3ea516edc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:18.338747 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2991aacc-b0b5-4e1f-a3b2-0b3ea516edc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:18.340322 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2991aacc-b0b5-4e1f-a3b2-0b3ea516edc8 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:18.350723 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1016/2081] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:14:18 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:14:18.745152 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:19.132427 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e34ca-3443-48c4-8ad4-44cbcfdc9c8c service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:19.167219 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:19.232199 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0e34ca-3443-48c4-8ad4-44cbcfdc9c8c service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:19.239554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d162097-9040-47c4-8b5c-17037cb7fcc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:19.240833 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d162097-9040-47c4-8b5c-17037cb7fcc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:19.241298 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d162097-9040-47c4-8b5c-17037cb7fcc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:19.241764 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d162097-9040-47c4-8b5c-17037cb7fcc5 admin admin] PATH_INFO: `/v3/users/ff36048f14214b779711e7230b51f426` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:19.244206 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d162097-9040-47c4-8b5c-17037cb7fcc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff36048f14214b779711e7230b51f426)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:19.253615 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d162097-9040-47c4-8b5c-17037cb7fcc5 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:19.523650 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1d162097-9040-47c4-8b5c-17037cb7fcc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff36048f14214b779711e7230b51f426'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:19.539221 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1d162097-9040-47c4-8b5c-17037cb7fcc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff36048f14214b779711e7230b51f426'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:19.539221 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1d162097-9040-47c4-8b5c-17037cb7fcc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff36048f14214b779711e7230b51f426'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:19.550004 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1d162097-9040-47c4-8b5c-17037cb7fcc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff36048f14214b779711e7230b51f426'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:19.653738 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1d162097-9040-47c4-8b5c-17037cb7fcc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff36048f14214b779711e7230b51f426'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:19.663635 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1066/2082] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:14:19 2022] DELETE /identity/v3/users/ff36048f14214b779711e7230b51f426 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:19.678311 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e4ba2-4d84-4ead-8c2a-18ef89ef6135 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:19.678761 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e4ba2-4d84-4ead-8c2a-18ef89ef6135 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:19.678918 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e4ba2-4d84-4ead-8c2a-18ef89ef6135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:19.701357 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-dd7e4ba2-4d84-4ead-8c2a-18ef89ef6135 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:19.703186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-dd7e4ba2-4d84-4ead-8c2a-18ef89ef6135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:19.794894 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1017/2083] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:19.806730 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d162097-9040-47c4-8b5c-17037cb7fcc5 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:19.808567 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-1d162097-9040-47c4-8b5c-17037cb7fcc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:19.940606 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d162097-9040-47c4-8b5c-17037cb7fcc5 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:19.945801 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7fad9e-95b8-4697-9deb-ee7dfd7dc1e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:19.946652 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7fad9e-95b8-4697-9deb-ee7dfd7dc1e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:19.946930 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7fad9e-95b8-4697-9deb-ee7dfd7dc1e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:19.947497 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7fad9e-95b8-4697-9deb-ee7dfd7dc1e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:19.971343 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7fad9e-95b8-4697-9deb-ee7dfd7dc1e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:19.973129 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7fad9e-95b8-4697-9deb-ee7dfd7dc1e4 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:19.994483 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1067/2084] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:14:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:14:20.077197 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:20.444391 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:20.809311 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e4ba2-4d84-4ead-8c2a-18ef89ef6135 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:20.811559 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-dd7e4ba2-4d84-4ead-8c2a-18ef89ef6135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:20.836438 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e4ba2-4d84-4ead-8c2a-18ef89ef6135 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:20.846403 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f238da7d-7872-44eb-806e-3a4ec18a32ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:20.847385 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238da7d-7872-44eb-806e-3a4ec18a32ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:20.848093 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238da7d-7872-44eb-806e-3a4ec18a32ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:20.848421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f238da7d-7872-44eb-806e-3a4ec18a32ed admin admin] PATH_INFO: `/v3/projects/e8c99d58ff2644599d70789319ec850a` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:20.849783 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f238da7d-7872-44eb-806e-3a4ec18a32ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8c99d58ff2644599d70789319ec850a)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:20.865153 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f238da7d-7872-44eb-806e-3a4ec18a32ed admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:21.061402 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-f238da7d-7872-44eb-806e-3a4ec18a32ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8c99d58ff2644599d70789319ec850a', 'request_id': 'req-f238da7d-7872-44eb-806e-3a4ec18a32ed'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:21.070216 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-f238da7d-7872-44eb-806e-3a4ec18a32ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8c99d58ff2644599d70789319ec850a', 'request_id': 'req-f238da7d-7872-44eb-806e-3a4ec18a32ed'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:21.180857 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-f238da7d-7872-44eb-806e-3a4ec18a32ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8c99d58ff2644599d70789319ec850a', 'request_id': 'req-f238da7d-7872-44eb-806e-3a4ec18a32ed'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:21.198240 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-f238da7d-7872-44eb-806e-3a4ec18a32ed admin admin] The token cache is being invalidate because project e8c99d58ff2644599d70789319ec850a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:14:21.198708 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-f238da7d-7872-44eb-806e-3a4ec18a32ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:21.201716 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-f238da7d-7872-44eb-806e-3a4ec18a32ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:21.207493 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1018/2085] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:14:20 2022] DELETE /identity/v3/projects/e8c99d58ff2644599d70789319ec850a => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:21.211510 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7fad9e-95b8-4697-9deb-ee7dfd7dc1e4 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:21.233403 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7fad9e-95b8-4697-9deb-ee7dfd7dc1e4 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:21.275597 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f238da7d-7872-44eb-806e-3a4ec18a32ed admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:21.277758 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-f238da7d-7872-44eb-806e-3a4ec18a32ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:21.352495 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd4b30b-c671-4700-8aba-3e6c874c9449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:21.353627 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd4b30b-c671-4700-8aba-3e6c874c9449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:21.354153 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd4b30b-c671-4700-8aba-3e6c874c9449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:21.354529 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd4b30b-c671-4700-8aba-3e6c874c9449 admin admin] PATH_INFO: `/v3/users/d2aec9298c524d59a159144d88fd5486` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:21.355962 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd4b30b-c671-4700-8aba-3e6c874c9449 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2aec9298c524d59a159144d88fd5486)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:21.358152 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd4b30b-c671-4700-8aba-3e6c874c9449 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:21.381081 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f238da7d-7872-44eb-806e-3a4ec18a32ed admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:21.386092 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f1cdb9-60b6-4ba9-b50b-7558a068dcd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:21.387054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1cdb9-60b6-4ba9-b50b-7558a068dcd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:21.387273 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1cdb9-60b6-4ba9-b50b-7558a068dcd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:21.388266 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1cdb9-60b6-4ba9-b50b-7558a068dcd6 admin admin] PATH_INFO: `/v3/users/0d023aca8ffd4e0a82565cae94cab53b` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:21.390087 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f1cdb9-60b6-4ba9-b50b-7558a068dcd6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d023aca8ffd4e0a82565cae94cab53b)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:21.392354 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f1cdb9-60b6-4ba9-b50b-7558a068dcd6 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:21.613888 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:21.657550 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-c5f1cdb9-60b6-4ba9-b50b-7558a068dcd6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d023aca8ffd4e0a82565cae94cab53b'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:21.660728 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-efd4b30b-c671-4700-8aba-3e6c874c9449 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2aec9298c524d59a159144d88fd5486'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:21.670419 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-efd4b30b-c671-4700-8aba-3e6c874c9449 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2aec9298c524d59a159144d88fd5486'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:21.671230 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-efd4b30b-c671-4700-8aba-3e6c874c9449 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2aec9298c524d59a159144d88fd5486'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:21.689033 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-c5f1cdb9-60b6-4ba9-b50b-7558a068dcd6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d023aca8ffd4e0a82565cae94cab53b'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:21.696293 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-efd4b30b-c671-4700-8aba-3e6c874c9449 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2aec9298c524d59a159144d88fd5486'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:21.707343 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-c5f1cdb9-60b6-4ba9-b50b-7558a068dcd6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d023aca8ffd4e0a82565cae94cab53b'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:21.708572 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-c5f1cdb9-60b6-4ba9-b50b-7558a068dcd6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d023aca8ffd4e0a82565cae94cab53b'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:21.867970 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-efd4b30b-c671-4700-8aba-3e6c874c9449 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2aec9298c524d59a159144d88fd5486'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:21.876401 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1068/2086] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:14:21 2022] DELETE /identity/v3/users/d2aec9298c524d59a159144d88fd5486 => generated 0 bytes in 665 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:21.877877 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd4b30b-c671-4700-8aba-3e6c874c9449 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:21.879507 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-efd4b30b-c671-4700-8aba-3e6c874c9449 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:21.928156 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-c5f1cdb9-60b6-4ba9-b50b-7558a068dcd6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d023aca8ffd4e0a82565cae94cab53b'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:21.940187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1019/2087] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:14:21 2022] DELETE /identity/v3/users/0d023aca8ffd4e0a82565cae94cab53b => generated 0 bytes in 665 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:21.944729 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b65c7e-af7a-4032-8977-99c4d6e582bf None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:21.951159 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b65c7e-af7a-4032-8977-99c4d6e582bf None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:21.951159 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b65c7e-af7a-4032-8977-99c4d6e582bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:22.014127 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd4b30b-c671-4700-8aba-3e6c874c9449 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:22.036962 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-a6b65c7e-af7a-4032-8977-99c4d6e582bf None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:22.042884 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-a6b65c7e-af7a-4032-8977-99c4d6e582bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:22.120287 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33af170-a33b-42f7-bbd8-8a0cccaa36b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:22.121233 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33af170-a33b-42f7-bbd8-8a0cccaa36b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:22.133061 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33af170-a33b-42f7-bbd8-8a0cccaa36b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:22.133061 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33af170-a33b-42f7-bbd8-8a0cccaa36b4 admin admin] PATH_INFO: `/v3/users/6c04736ff6a74cc3b228236efee80185` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:22.133061 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33af170-a33b-42f7-bbd8-8a0cccaa36b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c04736ff6a74cc3b228236efee80185)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:22.133061 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33af170-a33b-42f7-bbd8-8a0cccaa36b4 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:22.149970 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1020/2088] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:14:22.165420 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b65c7e-af7a-4032-8977-99c4d6e582bf None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:22.167373 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-a6b65c7e-af7a-4032-8977-99c4d6e582bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:22.180423 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:22.251035 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b65c7e-af7a-4032-8977-99c4d6e582bf None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:22.255364 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2adef4e1-3cb1-473a-bf05-34628047025c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:22.256064 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adef4e1-3cb1-473a-bf05-34628047025c service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:22.256594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adef4e1-3cb1-473a-bf05-34628047025c service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:22.256874 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2adef4e1-3cb1-473a-bf05-34628047025c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:22.271352 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adef4e1-3cb1-473a-bf05-34628047025c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:22.273314 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2adef4e1-3cb1-473a-bf05-34628047025c service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:22.288041 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1021/2089] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:14:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:14:22.348447 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f33af170-a33b-42f7-bbd8-8a0cccaa36b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c04736ff6a74cc3b228236efee80185'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:22.359001 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f33af170-a33b-42f7-bbd8-8a0cccaa36b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c04736ff6a74cc3b228236efee80185'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:22.360532 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f33af170-a33b-42f7-bbd8-8a0cccaa36b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c04736ff6a74cc3b228236efee80185'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:22.381350 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f33af170-a33b-42f7-bbd8-8a0cccaa36b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c04736ff6a74cc3b228236efee80185'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:22.504822 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f33af170-a33b-42f7-bbd8-8a0cccaa36b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c04736ff6a74cc3b228236efee80185'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:22.514237 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1069/2090] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:14:21 2022] DELETE /identity/v3/users/6c04736ff6a74cc3b228236efee80185 => generated 0 bytes in 637 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:22.519770 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a326c-c977-4d7b-895f-a529a081d63a None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:22.520061 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a326c-c977-4d7b-895f-a529a081d63a None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:22.520354 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a326c-c977-4d7b-895f-a529a081d63a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:22.552133 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-c77a326c-c977-4d7b-895f-a529a081d63a None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:22.553386 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c77a326c-c977-4d7b-895f-a529a081d63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:22.674166 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1022/2091] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:22.695349 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33af170-a33b-42f7-bbd8-8a0cccaa36b4 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:22.698011 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f33af170-a33b-42f7-bbd8-8a0cccaa36b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:22.782422 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33af170-a33b-42f7-bbd8-8a0cccaa36b4 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:22.786924 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcd83ba-59d7-443f-8f2c-173b10bffbda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:22.787602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd83ba-59d7-443f-8f2c-173b10bffbda service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:22.787944 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd83ba-59d7-443f-8f2c-173b10bffbda service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:22.788229 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcd83ba-59d7-443f-8f2c-173b10bffbda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:22.799012 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcd83ba-59d7-443f-8f2c-173b10bffbda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:22.800637 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcd83ba-59d7-443f-8f2c-173b10bffbda service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:22.807132 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77a326c-c977-4d7b-895f-a529a081d63a None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:22.808791 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c77a326c-c977-4d7b-895f-a529a081d63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:22.812998 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1070/2092] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:14:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:14:22.830511 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77a326c-c977-4d7b-895f-a529a081d63a None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:22.845200 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c0cef6-4630-4f21-a4e7-40bfdf540870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:22.847005 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0cef6-4630-4f21-a4e7-40bfdf540870 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:22.847612 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0cef6-4630-4f21-a4e7-40bfdf540870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:22.848099 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c0cef6-4630-4f21-a4e7-40bfdf540870 admin admin] PATH_INFO: `/v3/projects/afd9750067ed47bd8d83126b280d5b32` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:22.849529 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c0cef6-4630-4f21-a4e7-40bfdf540870 admin admin] RBAC: Authorizing `identity:delete_project(project_id=afd9750067ed47bd8d83126b280d5b32)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:22.851351 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c0cef6-4630-4f21-a4e7-40bfdf540870 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:23.037862 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-c8c0cef6-4630-4f21-a4e7-40bfdf540870 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afd9750067ed47bd8d83126b280d5b32', 'request_id': 'req-c8c0cef6-4630-4f21-a4e7-40bfdf540870'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:23.054473 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-c8c0cef6-4630-4f21-a4e7-40bfdf540870 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afd9750067ed47bd8d83126b280d5b32', 'request_id': 'req-c8c0cef6-4630-4f21-a4e7-40bfdf540870'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:23.127072 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-c8c0cef6-4630-4f21-a4e7-40bfdf540870 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afd9750067ed47bd8d83126b280d5b32', 'request_id': 'req-c8c0cef6-4630-4f21-a4e7-40bfdf540870'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:23.144246 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-c8c0cef6-4630-4f21-a4e7-40bfdf540870 admin admin] The token cache is being invalidate because project afd9750067ed47bd8d83126b280d5b32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:14:23.144541 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-c8c0cef6-4630-4f21-a4e7-40bfdf540870 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:23.145421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-c8c0cef6-4630-4f21-a4e7-40bfdf540870 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:23.147490 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1023/2093] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:14:22 2022] DELETE /identity/v3/projects/afd9750067ed47bd8d83126b280d5b32 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:23.153915 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e568b0e-2799-41b1-8cd3-b51f627da9e3 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:23.154418 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e568b0e-2799-41b1-8cd3-b51f627da9e3 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:23.154912 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e568b0e-2799-41b1-8cd3-b51f627da9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:23.187534 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-3e568b0e-2799-41b1-8cd3-b51f627da9e3 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:23.189358 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-3e568b0e-2799-41b1-8cd3-b51f627da9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:23.291215 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1071/2094] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:23.293871 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8136b2c-6425-4a0a-9ab5-43a751cd7d52 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:23.297504 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8136b2c-6425-4a0a-9ab5-43a751cd7d52 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:23.297504 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8136b2c-6425-4a0a-9ab5-43a751cd7d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:23.318772 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-d8136b2c-6425-4a0a-9ab5-43a751cd7d52 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:23.320466 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d8136b2c-6425-4a0a-9ab5-43a751cd7d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:23.331493 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1024/2095] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:23.335584 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e568b0e-2799-41b1-8cd3-b51f627da9e3 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:23.346811 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e568b0e-2799-41b1-8cd3-b51f627da9e3 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:23.352660 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97e9ee7-e17f-4f06-aa3e-4d036be68de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:23.353476 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e9ee7-e17f-4f06-aa3e-4d036be68de8 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:23.354075 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e9ee7-e17f-4f06-aa3e-4d036be68de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:23.354626 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97e9ee7-e17f-4f06-aa3e-4d036be68de8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:23.355542 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97e9ee7-e17f-4f06-aa3e-4d036be68de8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:14:23.356035 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97e9ee7-e17f-4f06-aa3e-4d036be68de8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:23.357857 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97e9ee7-e17f-4f06-aa3e-4d036be68de8 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:23.371390 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1072/2096] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:14:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:14:23.375464 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8136b2c-6425-4a0a-9ab5-43a751cd7d52 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:23.384124 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97e9ee7-e17f-4f06-aa3e-4d036be68de8 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:23.386028 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d97e9ee7-e17f-4f06-aa3e-4d036be68de8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:23.406792 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8136b2c-6425-4a0a-9ab5-43a751cd7d52 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:23.410972 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97e9ee7-e17f-4f06-aa3e-4d036be68de8 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:23.416702 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb7002f-2db5-42ec-8f36-34ec6b6651de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:23.417657 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7002f-2db5-42ec-8f36-34ec6b6651de admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:23.417992 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7002f-2db5-42ec-8f36-34ec6b6651de admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:23.418315 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb7002f-2db5-42ec-8f36-34ec6b6651de admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:23.419615 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb7002f-2db5-42ec-8f36-34ec6b6651de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:23.420695 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f607b65-e34b-4487-8195-e44371f8c711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:23.421796 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f607b65-e34b-4487-8195-e44371f8c711 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:23.422265 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f607b65-e34b-4487-8195-e44371f8c711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:23.422373 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb7002f-2db5-42ec-8f36-34ec6b6651de admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:23.422829 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f607b65-e34b-4487-8195-e44371f8c711 admin admin] PATH_INFO: `/v3/projects/b5664526f372472da13385845239c1a3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:23.424466 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f607b65-e34b-4487-8195-e44371f8c711 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5664526f372472da13385845239c1a3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:23.427337 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f607b65-e34b-4487-8195-e44371f8c711 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:23.484122 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:23.531203 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1025/2097] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:14:23 2022] POST /identity/v3/projects => generated 373 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:14:23.532097 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb7002f-2db5-42ec-8f36-34ec6b6651de admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:23.534073 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-6fb7002f-2db5-42ec-8f36-34ec6b6651de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:23.581667 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0f607b65-e34b-4487-8195-e44371f8c711 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5664526f372472da13385845239c1a3', 'request_id': 'req-0f607b65-e34b-4487-8195-e44371f8c711'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:23.588442 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0f607b65-e34b-4487-8195-e44371f8c711 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5664526f372472da13385845239c1a3', 'request_id': 'req-0f607b65-e34b-4487-8195-e44371f8c711'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:23.594548 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0f607b65-e34b-4487-8195-e44371f8c711 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5664526f372472da13385845239c1a3', 'request_id': 'req-0f607b65-e34b-4487-8195-e44371f8c711'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:23.601545 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb7002f-2db5-42ec-8f36-34ec6b6651de admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:23.647566 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0f607b65-e34b-4487-8195-e44371f8c711 admin admin] The token cache is being invalidate because project b5664526f372472da13385845239c1a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:14:23.647919 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0f607b65-e34b-4487-8195-e44371f8c711 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:23.648622 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0f607b65-e34b-4487-8195-e44371f8c711 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:23.650809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1073/2098] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:14:23 2022] DELETE /identity/v3/projects/b5664526f372472da13385845239c1a3 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:14:23.652534 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9657d4-f130-4acf-839c-7cceb19e36bf None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:23.652832 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9657d4-f130-4acf-839c-7cceb19e36bf None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:23.653762 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9657d4-f130-4acf-839c-7cceb19e36bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:23.658976 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ca053-9c53-4029-aebf-39914112cc12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:23.659807 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ca053-9c53-4029-aebf-39914112cc12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:23.660146 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ca053-9c53-4029-aebf-39914112cc12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:23.660422 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ca053-9c53-4029-aebf-39914112cc12 admin admin] PATH_INFO: `/v3/projects/c9ee5cf1e0ce461c82b1e37a5a222650` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:23.661705 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6ca053-9c53-4029-aebf-39914112cc12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9ee5cf1e0ce461c82b1e37a5a222650)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:23.663610 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6ca053-9c53-4029-aebf-39914112cc12 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:23.681121 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-4e9657d4-f130-4acf-839c-7cceb19e36bf None None] MFA Rules not processed for user `eb0e1e84f287480880a3150164e5a467`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:23.682377 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-4e9657d4-f130-4acf-839c-7cceb19e36bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:23.755668 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-fc6ca053-9c53-4029-aebf-39914112cc12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9ee5cf1e0ce461c82b1e37a5a222650', 'request_id': 'req-fc6ca053-9c53-4029-aebf-39914112cc12'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:23.757189 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1074/2099] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:23 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:23.758410 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9657d4-f130-4acf-839c-7cceb19e36bf None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:23.759293 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-4e9657d4-f130-4acf-839c-7cceb19e36bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:23.763556 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-fc6ca053-9c53-4029-aebf-39914112cc12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9ee5cf1e0ce461c82b1e37a5a222650', 'request_id': 'req-fc6ca053-9c53-4029-aebf-39914112cc12'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:23.770609 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:23.775155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9657d4-f130-4acf-839c-7cceb19e36bf None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:23.830720 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464f029-abc5-4d7a-97d8-51a6a36298e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:23.831394 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464f029-abc5-4d7a-97d8-51a6a36298e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:23.831679 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464f029-abc5-4d7a-97d8-51a6a36298e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:23.831984 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464f029-abc5-4d7a-97d8-51a6a36298e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:23.832673 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6464f029-abc5-4d7a-97d8-51a6a36298e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:23.834605 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6464f029-abc5-4d7a-97d8-51a6a36298e2 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:23.846411 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-fc6ca053-9c53-4029-aebf-39914112cc12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9ee5cf1e0ce461c82b1e37a5a222650', 'request_id': 'req-fc6ca053-9c53-4029-aebf-39914112cc12'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:23.858109 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-fc6ca053-9c53-4029-aebf-39914112cc12 admin admin] The token cache is being invalidate because project c9ee5cf1e0ce461c82b1e37a5a222650 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:14:23.858519 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-fc6ca053-9c53-4029-aebf-39914112cc12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:23.859243 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-fc6ca053-9c53-4029-aebf-39914112cc12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:23.861075 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1026/2100] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:14:23 2022] DELETE /identity/v3/projects/c9ee5cf1e0ce461c82b1e37a5a222650 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:23.865466 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ca053-9c53-4029-aebf-39914112cc12 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:23.867057 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-fc6ca053-9c53-4029-aebf-39914112cc12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:23.881318 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ca053-9c53-4029-aebf-39914112cc12 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:23.888273 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e003eb-bfe4-4377-b067-0c5e5042db84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:23.889197 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e003eb-bfe4-4377-b067-0c5e5042db84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:23.889476 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e003eb-bfe4-4377-b067-0c5e5042db84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:23.889937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e003eb-bfe4-4377-b067-0c5e5042db84 admin admin] PATH_INFO: `/v3/users/3d7b125a10cb4ad889fd3b72b13ccdf9` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:23.891566 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e003eb-bfe4-4377-b067-0c5e5042db84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d7b125a10cb4ad889fd3b72b13ccdf9)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:23.894759 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e003eb-bfe4-4377-b067-0c5e5042db84 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:23.945496 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1075/2101] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:14:23 2022] POST /identity/v3/users => generated 287 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:14:23.945496 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464f029-abc5-4d7a-97d8-51a6a36298e2 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:23.947567 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-6464f029-abc5-4d7a-97d8-51a6a36298e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:24.040618 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464f029-abc5-4d7a-97d8-51a6a36298e2 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:24.046942 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60721f53-270a-4fee-bb91-8d407c40c00f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:24.047820 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60721f53-270a-4fee-bb91-8d407c40c00f service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:24.048125 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60721f53-270a-4fee-bb91-8d407c40c00f service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:24.048390 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60721f53-270a-4fee-bb91-8d407c40c00f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:24.053434 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-60721f53-270a-4fee-bb91-8d407c40c00f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:24.090867 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-b4e003eb-bfe4-4377-b067-0c5e5042db84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d7b125a10cb4ad889fd3b72b13ccdf9'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:24.110440 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-b4e003eb-bfe4-4377-b067-0c5e5042db84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d7b125a10cb4ad889fd3b72b13ccdf9'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:24.118699 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60721f53-270a-4fee-bb91-8d407c40c00f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:24.119101 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-b4e003eb-bfe4-4377-b067-0c5e5042db84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d7b125a10cb4ad889fd3b72b13ccdf9'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:24.119957 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-b4e003eb-bfe4-4377-b067-0c5e5042db84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d7b125a10cb4ad889fd3b72b13ccdf9'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:24.120393 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60721f53-270a-4fee-bb91-8d407c40c00f service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:24.131048 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1076/2102] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:14:23 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:14:24.131867 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60721f53-270a-4fee-bb91-8d407c40c00f service nova] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:24.133322 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-60721f53-270a-4fee-bb91-8d407c40c00f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:24.146988 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60721f53-270a-4fee-bb91-8d407c40c00f service nova] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:24.152564 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613a239c-385b-49cc-b0af-d65c3f75a15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:24.153397 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a239c-385b-49cc-b0af-d65c3f75a15c admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:24.154323 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a239c-385b-49cc-b0af-d65c3f75a15c admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:24.154323 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613a239c-385b-49cc-b0af-d65c3f75a15c admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:24.154999 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613a239c-385b-49cc-b0af-d65c3f75a15c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:24.156838 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613a239c-385b-49cc-b0af-d65c3f75a15c admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:24.167967 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1077/2103] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:14:24.173466 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613a239c-385b-49cc-b0af-d65c3f75a15c admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:24.175231 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-b4e003eb-bfe4-4377-b067-0c5e5042db84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d7b125a10cb4ad889fd3b72b13ccdf9'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:24.188215 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1027/2104] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:14:23 2022] DELETE /identity/v3/users/3d7b125a10cb4ad889fd3b72b13ccdf9 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:24.189383 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613a239c-385b-49cc-b0af-d65c3f75a15c admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:24.283944 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589a8b75-8009-4c67-81fd-35a96e2f35a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:24.284681 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a8b75-8009-4c67-81fd-35a96e2f35a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:24.285818 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a8b75-8009-4c67-81fd-35a96e2f35a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:24.286331 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589a8b75-8009-4c67-81fd-35a96e2f35a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:24.288562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589a8b75-8009-4c67-81fd-35a96e2f35a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:24.290344 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589a8b75-8009-4c67-81fd-35a96e2f35a6 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:24.301937 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1078/2105] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:24 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:14:24.306622 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e003eb-bfe4-4377-b067-0c5e5042db84 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:24.308160 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-b4e003eb-bfe4-4377-b067-0c5e5042db84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:24.322632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e003eb-bfe4-4377-b067-0c5e5042db84 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:24.327388 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de2373-617e-425e-ab65-b97c9293f133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:24.328441 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de2373-617e-425e-ab65-b97c9293f133 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:24.328624 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de2373-617e-425e-ab65-b97c9293f133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:24.328909 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8de2373-617e-425e-ab65-b97c9293f133 admin admin] PATH_INFO: `/v3/projects/5fa7c1b11a1a4294b4b11e2147ee9f97/users/327c6a10fcbc414fa873c3bcda78a48a/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:24.348473 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8de2373-617e-425e-ab65-b97c9293f133 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fa7c1b11a1a4294b4b11e2147ee9f97, user_id=327c6a10fcbc414fa873c3bcda78a48a, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:24.351560 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8de2373-617e-425e-ab65-b97c9293f133 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:24.463961 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1028/2106] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:14:24 2022] PUT /identity/v3/projects/5fa7c1b11a1a4294b4b11e2147ee9f97/users/327c6a10fcbc414fa873c3bcda78a48a/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:24.476776 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218402d3-cd16-422f-8a03-6f1acd5dcf62 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:24.476963 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218402d3-cd16-422f-8a03-6f1acd5dcf62 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:24.477248 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218402d3-cd16-422f-8a03-6f1acd5dcf62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:24.535979 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-218402d3-cd16-422f-8a03-6f1acd5dcf62 None None] MFA Rules not processed for user `327c6a10fcbc414fa873c3bcda78a48a`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:24.538966 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-218402d3-cd16-422f-8a03-6f1acd5dcf62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:24.661783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1079/2107] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:24.666135 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de2373-617e-425e-ab65-b97c9293f133 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:24.755535 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8de2373-617e-425e-ab65-b97c9293f133 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:24.764031 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218402d3-cd16-422f-8a03-6f1acd5dcf62 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:24.765188 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812694d3-0d54-46b0-a984-dc65b8d7391e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:24.766286 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-218402d3-cd16-422f-8a03-6f1acd5dcf62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:24.766799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812694d3-0d54-46b0-a984-dc65b8d7391e admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:24.767277 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812694d3-0d54-46b0-a984-dc65b8d7391e admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:24.767926 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812694d3-0d54-46b0-a984-dc65b8d7391e admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:24.768895 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812694d3-0d54-46b0-a984-dc65b8d7391e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:24.771574 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812694d3-0d54-46b0-a984-dc65b8d7391e admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:24.785860 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:24.789402 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218402d3-cd16-422f-8a03-6f1acd5dcf62 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:24.794437 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bf9007-32cd-4d9d-b072-71c1861dcf78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:24.796497 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf9007-32cd-4d9d-b072-71c1861dcf78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:24.796808 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf9007-32cd-4d9d-b072-71c1861dcf78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:24.797068 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf9007-32cd-4d9d-b072-71c1861dcf78 admin admin] PATH_INFO: `/v3/projects/069c8384756f4796a71f5df99f15a80f` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:24.798282 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bf9007-32cd-4d9d-b072-71c1861dcf78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=069c8384756f4796a71f5df99f15a80f)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:24.800242 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bf9007-32cd-4d9d-b072-71c1861dcf78 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:24.888116 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1029/2108] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:14:24 2022] POST /identity/v3/projects => generated 373 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:14:24.892628 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812694d3-0d54-46b0-a984-dc65b8d7391e admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:24.955186 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f7bf9007-32cd-4d9d-b072-71c1861dcf78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '069c8384756f4796a71f5df99f15a80f', 'request_id': 'req-f7bf9007-32cd-4d9d-b072-71c1861dcf78'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:24.961731 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f7bf9007-32cd-4d9d-b072-71c1861dcf78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '069c8384756f4796a71f5df99f15a80f', 'request_id': 'req-f7bf9007-32cd-4d9d-b072-71c1861dcf78'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:24.969769 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f7bf9007-32cd-4d9d-b072-71c1861dcf78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '069c8384756f4796a71f5df99f15a80f', 'request_id': 'req-f7bf9007-32cd-4d9d-b072-71c1861dcf78'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:24.986688 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812694d3-0d54-46b0-a984-dc65b8d7391e admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:24.997995 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:25.052235 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f25a6-9f6a-4950-9e20-9b6818fc9f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:25.052823 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f25a6-9f6a-4950-9e20-9b6818fc9f6f admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:25.053177 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f25a6-9f6a-4950-9e20-9b6818fc9f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:25.053662 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721f25a6-9f6a-4950-9e20-9b6818fc9f6f admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:25.054538 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721f25a6-9f6a-4950-9e20-9b6818fc9f6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:25.056756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721f25a6-9f6a-4950-9e20-9b6818fc9f6f admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:25.060371 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f7bf9007-32cd-4d9d-b072-71c1861dcf78 admin admin] The token cache is being invalidate because project 069c8384756f4796a71f5df99f15a80f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:14:25.064102 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f7bf9007-32cd-4d9d-b072-71c1861dcf78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:25.065239 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f7bf9007-32cd-4d9d-b072-71c1861dcf78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:25.068907 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1080/2109] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:14:24 2022] DELETE /identity/v3/projects/069c8384756f4796a71f5df99f15a80f => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:25.073862 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7f792-72b1-490d-86f8-0371e6059868 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:25.074431 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7f792-72b1-490d-86f8-0371e6059868 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:25.074581 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc7f792-72b1-490d-86f8-0371e6059868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:25.093494 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-ccc7f792-72b1-490d-86f8-0371e6059868 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:25.094726 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-ccc7f792-72b1-490d-86f8-0371e6059868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:25.105321 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1081/2110] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:25.114334 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b268fe6-a43b-4d70-a0e2-e35cc4c1fb31 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:25.114721 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b268fe6-a43b-4d70-a0e2-e35cc4c1fb31 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:25.114981 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b268fe6-a43b-4d70-a0e2-e35cc4c1fb31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:25.139912 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-9b268fe6-a43b-4d70-a0e2-e35cc4c1fb31 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:25.141210 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-9b268fe6-a43b-4d70-a0e2-e35cc4c1fb31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:25.152197 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1030/2111] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:14:24 2022] POST /identity/v3/users => generated 293 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:14:25.160688 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f25a6-9f6a-4950-9e20-9b6818fc9f6f admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:25.160688 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-721f25a6-9f6a-4950-9e20-9b6818fc9f6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:25.175295 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1082/2112] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:25.179651 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b268fe6-a43b-4d70-a0e2-e35cc4c1fb31 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:25.190226 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721f25a6-9f6a-4950-9e20-9b6818fc9f6f admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:25.192213 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b268fe6-a43b-4d70-a0e2-e35cc4c1fb31 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:25.198282 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebaeee2-7db4-41f1-9ad6-770885020c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:25.199220 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebaeee2-7db4-41f1-9ad6-770885020c9f admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:25.199681 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebaeee2-7db4-41f1-9ad6-770885020c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:25.200860 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb24e93-ee2c-4127-9a45-8dd622c46dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:25.201171 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebaeee2-7db4-41f1-9ad6-770885020c9f admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:25.201722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb24e93-ee2c-4127-9a45-8dd622c46dde admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:25.202078 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb24e93-ee2c-4127-9a45-8dd622c46dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:25.202586 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb24e93-ee2c-4127-9a45-8dd622c46dde admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:25.203013 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebaeee2-7db4-41f1-9ad6-770885020c9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:14:25.203466 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb24e93-ee2c-4127-9a45-8dd622c46dde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:25.204126 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebaeee2-7db4-41f1-9ad6-770885020c9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:25.205822 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb24e93-ee2c-4127-9a45-8dd622c46dde admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:25.206460 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebaeee2-7db4-41f1-9ad6-770885020c9f admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:25.221650 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1031/2113] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:14:25.223125 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb24e93-ee2c-4127-9a45-8dd622c46dde admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:25.234386 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1083/2114] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:14:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 17:14:25.234566 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb24e93-ee2c-4127-9a45-8dd622c46dde admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:25.235820 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebaeee2-7db4-41f1-9ad6-770885020c9f admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:25.243613 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b182d-846f-47db-ba29-5d76c841c46e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:25.244937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b182d-846f-47db-ba29-5d76c841c46e admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:25.245211 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b182d-846f-47db-ba29-5d76c841c46e admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:25.245562 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2b182d-846f-47db-ba29-5d76c841c46e admin admin] PATH_INFO: `/v3/projects/52f58bb5cda84744a9c1b85d7129c111/users/937651817fe84e36b086f65f70bddb66/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:25.247109 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebaeee2-7db4-41f1-9ad6-770885020c9f admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:25.253200 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec664328-3687-4f50-9b99-75fafbe5b99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:25.253508 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec664328-3687-4f50-9b99-75fafbe5b99c admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:25.258191 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec664328-3687-4f50-9b99-75fafbe5b99c admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:25.258191 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec664328-3687-4f50-9b99-75fafbe5b99c admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:25.258191 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec664328-3687-4f50-9b99-75fafbe5b99c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:25.258191 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec664328-3687-4f50-9b99-75fafbe5b99c admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:25.262689 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2b182d-846f-47db-ba29-5d76c841c46e admin admin] RBAC: Authorizing `identity:create_grant(project_id=52f58bb5cda84744a9c1b85d7129c111, user_id=937651817fe84e36b086f65f70bddb66, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:25.265519 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2b182d-846f-47db-ba29-5d76c841c46e admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:25.350860 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1032/2115] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:14:25 2022] PUT /identity/v3/projects/52f58bb5cda84744a9c1b85d7129c111/users/937651817fe84e36b086f65f70bddb66/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:14:25.354709 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b182d-846f-47db-ba29-5d76c841c46e admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:25.407470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1084/2116] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:14:25 2022] POST /identity/v3/projects => generated 385 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:14:25.411628 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec664328-3687-4f50-9b99-75fafbe5b99c admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:25.424058 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2b182d-846f-47db-ba29-5d76c841c46e admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:25.494012 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec664328-3687-4f50-9b99-75fafbe5b99c admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:25.500167 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9032838a-fd28-4e86-a1c4-7bf127d23fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:25.500968 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9032838a-fd28-4e86-a1c4-7bf127d23fd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:25.501320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9032838a-fd28-4e86-a1c4-7bf127d23fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:25.501981 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9032838a-fd28-4e86-a1c4-7bf127d23fd0 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:25.503427 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9032838a-fd28-4e86-a1c4-7bf127d23fd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:25.506109 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9032838a-fd28-4e86-a1c4-7bf127d23fd0 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:25.538251 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e0c94a-7756-44fb-b5ed-80138d5cb894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:25.538942 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0c94a-7756-44fb-b5ed-80138d5cb894 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:25.539249 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0c94a-7756-44fb-b5ed-80138d5cb894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:25.539520 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e0c94a-7756-44fb-b5ed-80138d5cb894 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:25.540304 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e0c94a-7756-44fb-b5ed-80138d5cb894 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:25.541941 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e0c94a-7756-44fb-b5ed-80138d5cb894 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:25.553230 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1033/2117] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:14:25.557961 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e0c94a-7756-44fb-b5ed-80138d5cb894 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:25.568650 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e0c94a-7756-44fb-b5ed-80138d5cb894 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:25.575492 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e0dcac-2379-4e8b-a939-95990432aa8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:25.576448 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0dcac-2379-4e8b-a939-95990432aa8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:25.576739 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0dcac-2379-4e8b-a939-95990432aa8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:25.577021 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0dcac-2379-4e8b-a939-95990432aa8f admin admin] PATH_INFO: `/v3/projects/52f58bb5cda84744a9c1b85d7129c111/users/937651817fe84e36b086f65f70bddb66/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:25.579174 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e0dcac-2379-4e8b-a939-95990432aa8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=52f58bb5cda84744a9c1b85d7129c111, user_id=937651817fe84e36b086f65f70bddb66, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:25.582794 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e0dcac-2379-4e8b-a939-95990432aa8f admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:25.604361 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1034/2118] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:14:25 2022] PUT /identity/v3/projects/52f58bb5cda84744a9c1b85d7129c111/users/937651817fe84e36b086f65f70bddb66/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:25.613135 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51feb03d-fa4c-4ea6-8408-135a96909efa None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:25.613420 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51feb03d-fa4c-4ea6-8408-135a96909efa None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:25.613729 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51feb03d-fa4c-4ea6-8408-135a96909efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:25.642233 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1085/2119] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:14:25 2022] POST /identity/v3/users => generated 293 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:14:25.646221 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9032838a-fd28-4e86-a1c4-7bf127d23fd0 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:25.657060 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9032838a-fd28-4e86-a1c4-7bf127d23fd0 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:25.659060 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-51feb03d-fa4c-4ea6-8408-135a96909efa None None] MFA Rules not processed for user `937651817fe84e36b086f65f70bddb66`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:25.660584 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-51feb03d-fa4c-4ea6-8408-135a96909efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:25.729927 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fdcead-57fb-40c2-b688-830aa9e84414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:25.731135 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fdcead-57fb-40c2-b688-830aa9e84414 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:25.731425 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fdcead-57fb-40c2-b688-830aa9e84414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:25.731706 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fdcead-57fb-40c2-b688-830aa9e84414 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:25.732560 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fdcead-57fb-40c2-b688-830aa9e84414 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:25.737495 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fdcead-57fb-40c2-b688-830aa9e84414 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:25.760527 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1086/2120] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:14:25.765367 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fdcead-57fb-40c2-b688-830aa9e84414 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:25.781810 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1035/2121] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:25 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:25.796812 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fdcead-57fb-40c2-b688-830aa9e84414 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:25.797674 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51feb03d-fa4c-4ea6-8408-135a96909efa None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:25.799418 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-51feb03d-fa4c-4ea6-8408-135a96909efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:25.803419 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc4aee-23a4-4f9d-8802-9ee2fb08facc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:25.804144 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc4aee-23a4-4f9d-8802-9ee2fb08facc admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:25.804494 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc4aee-23a4-4f9d-8802-9ee2fb08facc admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:25.804727 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc4aee-23a4-4f9d-8802-9ee2fb08facc admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:25.805485 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc4aee-23a4-4f9d-8802-9ee2fb08facc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:25.807253 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc4aee-23a4-4f9d-8802-9ee2fb08facc admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:25.824478 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1087/2122] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:14:25.830054 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc4aee-23a4-4f9d-8802-9ee2fb08facc admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:25.838282 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:25.846204 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc4aee-23a4-4f9d-8802-9ee2fb08facc admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:25.851494 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec541cbc-b979-4e06-a416-e2a18b50eb3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:25.853992 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec541cbc-b979-4e06-a416-e2a18b50eb3b admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:25.853992 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec541cbc-b979-4e06-a416-e2a18b50eb3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:25.853992 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec541cbc-b979-4e06-a416-e2a18b50eb3b admin admin] PATH_INFO: `/v3/projects/4bee457a673b424f8fabdcb88c4f1d4d/users/6fb86720d1cd4a70bec442ddbfb5c6a0/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:25.880246 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec541cbc-b979-4e06-a416-e2a18b50eb3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bee457a673b424f8fabdcb88c4f1d4d, user_id=6fb86720d1cd4a70bec442ddbfb5c6a0, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:25.883125 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec541cbc-b979-4e06-a416-e2a18b50eb3b admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:25.895795 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51feb03d-fa4c-4ea6-8408-135a96909efa None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:25.899774 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55390f00-0c6a-4c2f-b446-76d0d3cbc86e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:25.900504 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55390f00-0c6a-4c2f-b446-76d0d3cbc86e service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:25.900861 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55390f00-0c6a-4c2f-b446-76d0d3cbc86e service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:25.901227 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55390f00-0c6a-4c2f-b446-76d0d3cbc86e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:25.904138 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-55390f00-0c6a-4c2f-b446-76d0d3cbc86e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:25.964485 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1088/2123] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:14:25 2022] PUT /identity/v3/projects/4bee457a673b424f8fabdcb88c4f1d4d/users/6fb86720d1cd4a70bec442ddbfb5c6a0/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:25.978066 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55390f00-0c6a-4c2f-b446-76d0d3cbc86e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:25.979758 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55490bd-3698-41e2-82b9-bf2fbe30571d None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:25.979879 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55390f00-0c6a-4c2f-b446-76d0d3cbc86e service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:25.980113 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55490bd-3698-41e2-82b9-bf2fbe30571d None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:25.980352 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55490bd-3698-41e2-82b9-bf2fbe30571d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:25.991275 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1036/2124] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:14:25 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:14:26.041126 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-e55490bd-3698-41e2-82b9-bf2fbe30571d None None] MFA Rules not processed for user `6fb86720d1cd4a70bec442ddbfb5c6a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:26.042966 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-e55490bd-3698-41e2-82b9-bf2fbe30571d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:26.142731 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1089/2125] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:26.147178 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55390f00-0c6a-4c2f-b446-76d0d3cbc86e service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:26.160018 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55390f00-0c6a-4c2f-b446-76d0d3cbc86e service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:26.240278 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d693c-591a-4b2d-b47f-245adfb96a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:26.240607 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d693c-591a-4b2d-b47f-245adfb96a00 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:26.240947 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d693c-591a-4b2d-b47f-245adfb96a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:26.241205 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d693c-591a-4b2d-b47f-245adfb96a00 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:26.243286 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0d693c-591a-4b2d-b47f-245adfb96a00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:26.250581 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0d693c-591a-4b2d-b47f-245adfb96a00 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:26.373885 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1037/2126] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:14:26 2022] POST /identity/v3/projects => generated 385 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:14:26.378519 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55490bd-3698-41e2-82b9-bf2fbe30571d None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:26.390983 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55490bd-3698-41e2-82b9-bf2fbe30571d None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:26.467529 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78ca76-ac25-40fb-853d-ffef389b153d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:26.468535 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78ca76-ac25-40fb-853d-ffef389b153d admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:26.469246 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78ca76-ac25-40fb-853d-ffef389b153d admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:26.469729 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78ca76-ac25-40fb-853d-ffef389b153d admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:26.471271 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b78ca76-ac25-40fb-853d-ffef389b153d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:26.473855 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b78ca76-ac25-40fb-853d-ffef389b153d admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:26.525318 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:26.587346 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1090/2127] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:14:26 2022] POST /identity/v3/users => generated 293 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:14:26.591964 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d693c-591a-4b2d-b47f-245adfb96a00 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:26.615692 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d693c-591a-4b2d-b47f-245adfb96a00 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:26.629865 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361d1e09-37ea-42cf-90e5-a34aa5292565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:26.631908 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d1e09-37ea-42cf-90e5-a34aa5292565 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:26.632768 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d1e09-37ea-42cf-90e5-a34aa5292565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:26.636905 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361d1e09-37ea-42cf-90e5-a34aa5292565 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:26.644533 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361d1e09-37ea-42cf-90e5-a34aa5292565 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:26.654171 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361d1e09-37ea-42cf-90e5-a34aa5292565 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:26.689484 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1038/2128] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:14:26.692321 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78ca76-ac25-40fb-853d-ffef389b153d admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:26.705713 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78ca76-ac25-40fb-853d-ffef389b153d admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:26.741657 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617895ed-26cd-4586-8232-d42fe492e43a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:26.742679 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617895ed-26cd-4586-8232-d42fe492e43a admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:26.743094 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617895ed-26cd-4586-8232-d42fe492e43a admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:26.743617 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617895ed-26cd-4586-8232-d42fe492e43a admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:26.744664 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617895ed-26cd-4586-8232-d42fe492e43a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:26.750254 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617895ed-26cd-4586-8232-d42fe492e43a admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:26.766083 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1091/2129] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:26 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:14:26.770115 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361d1e09-37ea-42cf-90e5-a34aa5292565 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:26.783987 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361d1e09-37ea-42cf-90e5-a34aa5292565 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:26.789129 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee290cb-1d76-4f99-932a-c1e8c1b41258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:26.790045 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee290cb-1d76-4f99-932a-c1e8c1b41258 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:26.790326 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee290cb-1d76-4f99-932a-c1e8c1b41258 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:26.790663 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee290cb-1d76-4f99-932a-c1e8c1b41258 admin admin] PATH_INFO: `/v3/projects/8c34969de69a4287a65ff0f4fe026b75/users/bf9b62c9870d435f90d1ef177d6d0dc7/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:26.808158 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee290cb-1d76-4f99-932a-c1e8c1b41258 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c34969de69a4287a65ff0f4fe026b75, user_id=bf9b62c9870d435f90d1ef177d6d0dc7, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:26.811168 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee290cb-1d76-4f99-932a-c1e8c1b41258 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:26.929135 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1039/2130] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:14:26 2022] PUT /identity/v3/projects/8c34969de69a4287a65ff0f4fe026b75/users/bf9b62c9870d435f90d1ef177d6d0dc7/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:14:26.941240 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7970f726-c5b9-4948-aab6-c384499b5668 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:26.941667 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7970f726-c5b9-4948-aab6-c384499b5668 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:26.941954 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7970f726-c5b9-4948-aab6-c384499b5668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:26.982086 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:26.983439 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-7970f726-c5b9-4948-aab6-c384499b5668 None None] MFA Rules not processed for user `bf9b62c9870d435f90d1ef177d6d0dc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:26.985410 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-7970f726-c5b9-4948-aab6-c384499b5668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:27.064538 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1092/2131] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:27.068822 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee290cb-1d76-4f99-932a-c1e8c1b41258 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:27.150494 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee290cb-1d76-4f99-932a-c1e8c1b41258 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:27.155978 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fcf8d1-c648-4d5f-8641-40cdea4f8a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:27.156658 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fcf8d1-c648-4d5f-8641-40cdea4f8a92 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:27.156944 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fcf8d1-c648-4d5f-8641-40cdea4f8a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:27.157221 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fcf8d1-c648-4d5f-8641-40cdea4f8a92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:27.157914 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fcf8d1-c648-4d5f-8641-40cdea4f8a92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:27.160000 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fcf8d1-c648-4d5f-8641-40cdea4f8a92 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:27.260869 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1040/2132] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:14:27 2022] POST /identity/v3/projects => generated 383 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:14:27.262459 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7970f726-c5b9-4948-aab6-c384499b5668 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:27.285263 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7970f726-c5b9-4948-aab6-c384499b5668 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:27.348108 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e1801-d119-444e-890b-a4cacec0a538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:27.348805 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e1801-d119-444e-890b-a4cacec0a538 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:27.349093 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e1801-d119-444e-890b-a4cacec0a538 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:27.349371 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e1801-d119-444e-890b-a4cacec0a538 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:27.350079 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33e1801-d119-444e-890b-a4cacec0a538 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:27.352062 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33e1801-d119-444e-890b-a4cacec0a538 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:27.379281 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:27.381473 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3331b52-7799-4397-9698-bd653240fec7 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:27.381768 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3331b52-7799-4397-9698-bd653240fec7 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:27.382052 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3331b52-7799-4397-9698-bd653240fec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:27.400207 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-c3331b52-7799-4397-9698-bd653240fec7 None None] MFA Rules not processed for user `2601402e569e491a94cea7a312cfef86`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:27.401681 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c3331b52-7799-4397-9698-bd653240fec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:27.459180 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1041/2133] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:14:27.549125 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1093/2134] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:14:27 2022] POST /identity/v3/users => generated 298 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:14:27.553104 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3331b52-7799-4397-9698-bd653240fec7 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:27.565836 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3331b52-7799-4397-9698-bd653240fec7 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:27.572180 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e05b34-7a64-4e55-8c7f-1aeba7e53ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:27.572925 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e05b34-7a64-4e55-8c7f-1aeba7e53ca8 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:27.573216 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e05b34-7a64-4e55-8c7f-1aeba7e53ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:27.573753 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e05b34-7a64-4e55-8c7f-1aeba7e53ca8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:27.575936 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e05b34-7a64-4e55-8c7f-1aeba7e53ca8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:27.577035 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e05b34-7a64-4e55-8c7f-1aeba7e53ca8 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:27.594849 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1042/2135] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:14:27.599048 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e1801-d119-444e-890b-a4cacec0a538 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:27.610386 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e1801-d119-444e-890b-a4cacec0a538 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:27.615378 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdaaeb8-266d-4821-ac5d-95bbb7eb8aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:27.616464 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdaaeb8-266d-4821-ac5d-95bbb7eb8aa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:27.616815 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdaaeb8-266d-4821-ac5d-95bbb7eb8aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:27.617204 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdaaeb8-266d-4821-ac5d-95bbb7eb8aa3 admin admin] PATH_INFO: `/v3/projects/6c029de4dda94aabba4ab5c0ddd45b90/users/eddd17f7c2864e4bb38b440d9ec62e75/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:27.641547 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdaaeb8-266d-4821-ac5d-95bbb7eb8aa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c029de4dda94aabba4ab5c0ddd45b90, user_id=eddd17f7c2864e4bb38b440d9ec62e75, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:27.644512 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdaaeb8-266d-4821-ac5d-95bbb7eb8aa3 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:27.769401 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1094/2136] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:14:27 2022] PUT /identity/v3/projects/6c029de4dda94aabba4ab5c0ddd45b90/users/eddd17f7c2864e4bb38b440d9ec62e75/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:27.780724 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e05b34-7a64-4e55-8c7f-1aeba7e53ca8 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:27.793370 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e05b34-7a64-4e55-8c7f-1aeba7e53ca8 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:27.852117 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca06b50-7feb-4205-892a-ad36589dbc8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:27.852887 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca06b50-7feb-4205-892a-ad36589dbc8c admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:27.853177 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca06b50-7feb-4205-892a-ad36589dbc8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:27.853453 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca06b50-7feb-4205-892a-ad36589dbc8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:27.854190 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca06b50-7feb-4205-892a-ad36589dbc8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:27.855811 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca06b50-7feb-4205-892a-ad36589dbc8c admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:27.865769 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1043/2137] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:27 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:14:27.870143 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdaaeb8-266d-4821-ac5d-95bbb7eb8aa3 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:27.885674 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdaaeb8-266d-4821-ac5d-95bbb7eb8aa3 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:27.892989 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d3dcfd-64ff-4642-a09c-ba53cf2f95e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:27.892989 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3dcfd-64ff-4642-a09c-ba53cf2f95e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:27.892989 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3dcfd-64ff-4642-a09c-ba53cf2f95e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:27.892989 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d3dcfd-64ff-4642-a09c-ba53cf2f95e4 admin admin] PATH_INFO: `/v3/projects/6c029de4dda94aabba4ab5c0ddd45b90/users/eddd17f7c2864e4bb38b440d9ec62e75/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:27.894957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d3dcfd-64ff-4642-a09c-ba53cf2f95e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c029de4dda94aabba4ab5c0ddd45b90, user_id=eddd17f7c2864e4bb38b440d9ec62e75, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:27.897678 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d3dcfd-64ff-4642-a09c-ba53cf2f95e4 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:27.909505 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1095/2138] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:14:27 2022] PUT /identity/v3/projects/6c029de4dda94aabba4ab5c0ddd45b90/users/eddd17f7c2864e4bb38b440d9ec62e75/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:27.921669 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198f352-4c1c-418e-a08d-46da1c4dc280 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:27.921969 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198f352-4c1c-418e-a08d-46da1c4dc280 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:27.922255 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198f352-4c1c-418e-a08d-46da1c4dc280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:27.951173 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-0198f352-4c1c-418e-a08d-46da1c4dc280 None None] MFA Rules not processed for user `eddd17f7c2864e4bb38b440d9ec62e75`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:27.951490 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-0198f352-4c1c-418e-a08d-46da1c4dc280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:28.034409 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1044/2139] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:27 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:14:28.264429 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d3dcfd-64ff-4642-a09c-ba53cf2f95e4 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:28.319344 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d3dcfd-64ff-4642-a09c-ba53cf2f95e4 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:28.323095 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d10d64-4d0f-4165-a032-b5a62d4dacf4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:28.323757 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d10d64-4d0f-4165-a032-b5a62d4dacf4 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:28.324050 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d10d64-4d0f-4165-a032-b5a62d4dacf4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:28.324326 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d10d64-4d0f-4165-a032-b5a62d4dacf4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:28.381188 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d10d64-4d0f-4165-a032-b5a62d4dacf4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:28.382632 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d10d64-4d0f-4165-a032-b5a62d4dacf4 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:28.391789 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1096/2140] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:14:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:14:28.803302 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0198f352-4c1c-418e-a08d-46da1c4dc280 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:28.870277 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0198f352-4c1c-418e-a08d-46da1c4dc280 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:28.875893 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69091537-e100-4027-9593-e50a53e8de60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:28.876830 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69091537-e100-4027-9593-e50a53e8de60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:28.878512 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69091537-e100-4027-9593-e50a53e8de60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:28.878512 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69091537-e100-4027-9593-e50a53e8de60 admin admin] PATH_INFO: `/v3/users/327c6a10fcbc414fa873c3bcda78a48a` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:28.878731 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69091537-e100-4027-9593-e50a53e8de60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=327c6a10fcbc414fa873c3bcda78a48a)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:28.880854 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69091537-e100-4027-9593-e50a53e8de60 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:29.114287 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-69091537-e100-4027-9593-e50a53e8de60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '327c6a10fcbc414fa873c3bcda78a48a'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:29.128298 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-69091537-e100-4027-9593-e50a53e8de60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '327c6a10fcbc414fa873c3bcda78a48a'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:29.135788 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-69091537-e100-4027-9593-e50a53e8de60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '327c6a10fcbc414fa873c3bcda78a48a'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:29.137543 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-69091537-e100-4027-9593-e50a53e8de60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '327c6a10fcbc414fa873c3bcda78a48a'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:29.196904 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:29.248040 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-69091537-e100-4027-9593-e50a53e8de60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '327c6a10fcbc414fa873c3bcda78a48a'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:29.256536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1045/2141] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:14:28 2022] DELETE /identity/v3/users/327c6a10fcbc414fa873c3bcda78a48a => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:29.261240 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14e42e-1893-43f9-8678-b515ab0b8fa0 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:29.261697 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14e42e-1893-43f9-8678-b515ab0b8fa0 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:29.261788 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf14e42e-1893-43f9-8678-b515ab0b8fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:29.317914 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-bf14e42e-1893-43f9-8678-b515ab0b8fa0 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:29.319198 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-bf14e42e-1893-43f9-8678-b515ab0b8fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:29.397576 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1097/2142] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:29.408955 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69091537-e100-4027-9593-e50a53e8de60 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:29.410396 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-69091537-e100-4027-9593-e50a53e8de60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:29.488876 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69091537-e100-4027-9593-e50a53e8de60 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:29.493359 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e91500-df9f-4658-bf50-90b512af8a5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:29.494288 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e91500-df9f-4658-bf50-90b512af8a5f service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:29.494637 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e91500-df9f-4658-bf50-90b512af8a5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:29.494989 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e91500-df9f-4658-bf50-90b512af8a5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:29.507232 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e91500-df9f-4658-bf50-90b512af8a5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:29.509215 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e91500-df9f-4658-bf50-90b512af8a5f service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:29.521212 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1046/2143] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:14:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:14:29.819601 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:30.142100 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf14e42e-1893-43f9-8678-b515ab0b8fa0 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:30.155863 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-bf14e42e-1893-43f9-8678-b515ab0b8fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:30.157804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf14e42e-1893-43f9-8678-b515ab0b8fa0 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:30.164252 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c547097-7572-467a-8449-e0f0a10a147f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:30.165169 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c547097-7572-467a-8449-e0f0a10a147f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:30.165457 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c547097-7572-467a-8449-e0f0a10a147f admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:30.165741 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c547097-7572-467a-8449-e0f0a10a147f admin admin] PATH_INFO: `/v3/projects/5fa7c1b11a1a4294b4b11e2147ee9f97` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:30.166967 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c547097-7572-467a-8449-e0f0a10a147f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fa7c1b11a1a4294b4b11e2147ee9f97)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:30.169465 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c547097-7572-467a-8449-e0f0a10a147f admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:30.335747 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1c547097-7572-467a-8449-e0f0a10a147f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fa7c1b11a1a4294b4b11e2147ee9f97', 'request_id': 'req-1c547097-7572-467a-8449-e0f0a10a147f'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:30.342192 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1c547097-7572-467a-8449-e0f0a10a147f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fa7c1b11a1a4294b4b11e2147ee9f97', 'request_id': 'req-1c547097-7572-467a-8449-e0f0a10a147f'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:30.347802 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1c547097-7572-467a-8449-e0f0a10a147f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fa7c1b11a1a4294b4b11e2147ee9f97', 'request_id': 'req-1c547097-7572-467a-8449-e0f0a10a147f'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:30.443894 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1c547097-7572-467a-8449-e0f0a10a147f admin admin] The token cache is being invalidate because project 5fa7c1b11a1a4294b4b11e2147ee9f97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:14:30.444620 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1c547097-7572-467a-8449-e0f0a10a147f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:30.445292 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1c547097-7572-467a-8449-e0f0a10a147f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:30.447345 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1098/2144] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:14:30 2022] DELETE /identity/v3/projects/5fa7c1b11a1a4294b4b11e2147ee9f97 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:30.451711 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e91500-df9f-4658-bf50-90b512af8a5f service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:30.461917 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e91500-df9f-4658-bf50-90b512af8a5f service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:30.520784 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6900ad2f-6b9a-4e46-9473-f0dd16721035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:30.521666 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6900ad2f-6b9a-4e46-9473-f0dd16721035 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:30.521948 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6900ad2f-6b9a-4e46-9473-f0dd16721035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:30.522225 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6900ad2f-6b9a-4e46-9473-f0dd16721035 admin admin] PATH_INFO: `/v3/users/937651817fe84e36b086f65f70bddb66` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:30.523398 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6900ad2f-6b9a-4e46-9473-f0dd16721035 admin admin] RBAC: Authorizing `identity:delete_user(user_id=937651817fe84e36b086f65f70bddb66)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:30.525348 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6900ad2f-6b9a-4e46-9473-f0dd16721035 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:30.680144 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c547097-7572-467a-8449-e0f0a10a147f admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:30.681956 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-1c547097-7572-467a-8449-e0f0a10a147f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:30.713649 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6900ad2f-6b9a-4e46-9473-f0dd16721035 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '937651817fe84e36b086f65f70bddb66'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:30.727433 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6900ad2f-6b9a-4e46-9473-f0dd16721035 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '937651817fe84e36b086f65f70bddb66'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:30.745516 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6900ad2f-6b9a-4e46-9473-f0dd16721035 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '937651817fe84e36b086f65f70bddb66'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:30.746420 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6900ad2f-6b9a-4e46-9473-f0dd16721035 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '937651817fe84e36b086f65f70bddb66'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:30.757846 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c547097-7572-467a-8449-e0f0a10a147f admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:30.763505 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3a31e5-337b-48e3-a3df-7b2bfa3b2c16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:30.763505 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a31e5-337b-48e3-a3df-7b2bfa3b2c16 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:30.763757 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a31e5-337b-48e3-a3df-7b2bfa3b2c16 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:30.763978 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3a31e5-337b-48e3-a3df-7b2bfa3b2c16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:30.765755 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-6f3a31e5-337b-48e3-a3df-7b2bfa3b2c16 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:30.819109 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6900ad2f-6b9a-4e46-9473-f0dd16721035 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '937651817fe84e36b086f65f70bddb66'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:30.827788 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1047/2145] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:14:30 2022] DELETE /identity/v3/users/937651817fe84e36b086f65f70bddb66 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:30.842750 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3a31e5-337b-48e3-a3df-7b2bfa3b2c16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:30.844268 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3a31e5-337b-48e3-a3df-7b2bfa3b2c16 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:30.854017 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1099/2146] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:14:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:14:31.266236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6900ad2f-6b9a-4e46-9473-f0dd16721035 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:31.267925 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-6900ad2f-6b9a-4e46-9473-f0dd16721035 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:31.348258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6900ad2f-6b9a-4e46-9473-f0dd16721035 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:31.351507 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:31.361072 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4c16e9-dc38-4cc1-b657-3ece7ce8ba15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:31.361953 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c16e9-dc38-4cc1-b657-3ece7ce8ba15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:31.362242 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c16e9-dc38-4cc1-b657-3ece7ce8ba15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:31.362538 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4c16e9-dc38-4cc1-b657-3ece7ce8ba15 admin admin] PATH_INFO: `/v3/projects/52f58bb5cda84744a9c1b85d7129c111` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:31.363765 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c16e9-dc38-4cc1-b657-3ece7ce8ba15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52f58bb5cda84744a9c1b85d7129c111)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:31.366063 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4c16e9-dc38-4cc1-b657-3ece7ce8ba15 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:31.477484 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0c4c16e9-dc38-4cc1-b657-3ece7ce8ba15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52f58bb5cda84744a9c1b85d7129c111', 'request_id': 'req-0c4c16e9-dc38-4cc1-b657-3ece7ce8ba15'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:31.486043 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0c4c16e9-dc38-4cc1-b657-3ece7ce8ba15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52f58bb5cda84744a9c1b85d7129c111', 'request_id': 'req-0c4c16e9-dc38-4cc1-b657-3ece7ce8ba15'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:31.539021 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0c4c16e9-dc38-4cc1-b657-3ece7ce8ba15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52f58bb5cda84744a9c1b85d7129c111', 'request_id': 'req-0c4c16e9-dc38-4cc1-b657-3ece7ce8ba15'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:31.550586 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0c4c16e9-dc38-4cc1-b657-3ece7ce8ba15 admin admin] The token cache is being invalidate because project 52f58bb5cda84744a9c1b85d7129c111 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:14:31.550881 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0c4c16e9-dc38-4cc1-b657-3ece7ce8ba15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:31.551537 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0c4c16e9-dc38-4cc1-b657-3ece7ce8ba15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:31.553203 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1048/2147] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:14:31 2022] DELETE /identity/v3/projects/52f58bb5cda84744a9c1b85d7129c111 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:31.557097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:31.558940 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb810f-a081-41ff-a6b8-a6204a1457dc None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:31.559422 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb810f-a081-41ff-a6b8-a6204a1457dc None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:31.559711 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedb810f-a081-41ff-a6b8-a6204a1457dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:31.580220 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-eedb810f-a081-41ff-a6b8-a6204a1457dc None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:31.585114 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-eedb810f-a081-41ff-a6b8-a6204a1457dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:31.645555 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1100/2148] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:31.654132 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebecb889-0355-4852-aefe-d61bfdd761e1 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:31.654448 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebecb889-0355-4852-aefe-d61bfdd761e1 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:31.654725 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebecb889-0355-4852-aefe-d61bfdd761e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:31.674172 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-ebecb889-0355-4852-aefe-d61bfdd761e1 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:31.675595 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ebecb889-0355-4852-aefe-d61bfdd761e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:31.683630 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1049/2149] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:31.688166 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb810f-a081-41ff-a6b8-a6204a1457dc None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:31.697711 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedb810f-a081-41ff-a6b8-a6204a1457dc None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:31.702851 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c79c245-2dee-49dd-9ba1-df4b790c8e38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:31.703502 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79c245-2dee-49dd-9ba1-df4b790c8e38 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:31.703827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79c245-2dee-49dd-9ba1-df4b790c8e38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:31.704110 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79c245-2dee-49dd-9ba1-df4b790c8e38 admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:31.704840 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c79c245-2dee-49dd-9ba1-df4b790c8e38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:14:31.705148 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c79c245-2dee-49dd-9ba1-df4b790c8e38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:31.706737 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c79c245-2dee-49dd-9ba1-df4b790c8e38 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:31.728615 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1101/2150] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:14:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:14:31.728823 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebecb889-0355-4852-aefe-d61bfdd761e1 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:31.738648 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebecb889-0355-4852-aefe-d61bfdd761e1 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:31.743289 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b33f4da-e4f0-42f7-a2ca-13c319a35450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:31.743983 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b33f4da-e4f0-42f7-a2ca-13c319a35450 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:31.744268 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b33f4da-e4f0-42f7-a2ca-13c319a35450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:31.744591 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b33f4da-e4f0-42f7-a2ca-13c319a35450 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:31.745277 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b33f4da-e4f0-42f7-a2ca-13c319a35450 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:31.747211 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b33f4da-e4f0-42f7-a2ca-13c319a35450 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:31.834547 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1050/2151] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:14:31 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:14:31.835756 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c79c245-2dee-49dd-9ba1-df4b790c8e38 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:31.845808 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c79c245-2dee-49dd-9ba1-df4b790c8e38 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:31.914582 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d295af47-7c5f-4b48-8bb9-c80962f36abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:31.915430 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295af47-7c5f-4b48-8bb9-c80962f36abd admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:31.915582 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295af47-7c5f-4b48-8bb9-c80962f36abd admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:31.915910 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d295af47-7c5f-4b48-8bb9-c80962f36abd admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:31.916606 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d295af47-7c5f-4b48-8bb9-c80962f36abd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:31.919155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d295af47-7c5f-4b48-8bb9-c80962f36abd admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:32.001896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1102/2152] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:14:31 2022] POST /identity/v3/users => generated 289 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:14:32.002049 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b33f4da-e4f0-42f7-a2ca-13c319a35450 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:32.012207 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b33f4da-e4f0-42f7-a2ca-13c319a35450 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:32.020264 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b0a14-c43d-4795-92b7-f4f2607f0aa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:32.020977 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b0a14-c43d-4795-92b7-f4f2607f0aa6 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:32.021805 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b0a14-c43d-4795-92b7-f4f2607f0aa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:32.021805 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b0a14-c43d-4795-92b7-f4f2607f0aa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:32.022740 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56b0a14-c43d-4795-92b7-f4f2607f0aa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:32.024052 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56b0a14-c43d-4795-92b7-f4f2607f0aa6 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:32.039343 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1051/2153] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:14:32.045917 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d295af47-7c5f-4b48-8bb9-c80962f36abd admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:32.059533 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d295af47-7c5f-4b48-8bb9-c80962f36abd admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:32.064614 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0031a59c-94d8-4607-8ee5-7c70e66558a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:32.065294 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031a59c-94d8-4607-8ee5-7c70e66558a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:32.065570 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031a59c-94d8-4607-8ee5-7c70e66558a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:32.065847 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0031a59c-94d8-4607-8ee5-7c70e66558a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:32.066629 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0031a59c-94d8-4607-8ee5-7c70e66558a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:32.068325 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0031a59c-94d8-4607-8ee5-7c70e66558a2 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:32.086118 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1103/2154] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:14:32.086655 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b0a14-c43d-4795-92b7-f4f2607f0aa6 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:32.100808 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b0a14-c43d-4795-92b7-f4f2607f0aa6 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:32.107754 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff1dd80-2f69-4485-941a-44a33f407c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:32.109811 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1dd80-2f69-4485-941a-44a33f407c6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:32.110115 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1dd80-2f69-4485-941a-44a33f407c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:32.110378 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff1dd80-2f69-4485-941a-44a33f407c6b admin admin] PATH_INFO: `/v3/projects/f27c0fbe98e7417487064594bbac7854/users/3fbc4348c7c647cc864f659e10477eab/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:32.135120 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0031a59c-94d8-4607-8ee5-7c70e66558a2 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:32.135543 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff1dd80-2f69-4485-941a-44a33f407c6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f27c0fbe98e7417487064594bbac7854, user_id=3fbc4348c7c647cc864f659e10477eab, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:32.136896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-0031a59c-94d8-4607-8ee5-7c70e66558a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:32.138834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff1dd80-2f69-4485-941a-44a33f407c6b admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:32.149780 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0031a59c-94d8-4607-8ee5-7c70e66558a2 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:32.155928 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c3743-54a4-4662-9a46-45d15b33bfd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:32.156861 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c3743-54a4-4662-9a46-45d15b33bfd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:32.157148 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c3743-54a4-4662-9a46-45d15b33bfd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:32.157423 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c3743-54a4-4662-9a46-45d15b33bfd4 admin admin] PATH_INFO: `/v3/users/780716e0cd604f2aa6ee9bd8b0ca645f` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:32.158777 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72c3743-54a4-4662-9a46-45d15b33bfd4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=780716e0cd604f2aa6ee9bd8b0ca645f)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:32.160804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72c3743-54a4-4662-9a46-45d15b33bfd4 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:32.285714 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1052/2155] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:14:32 2022] PUT /identity/v3/projects/f27c0fbe98e7417487064594bbac7854/users/3fbc4348c7c647cc864f659e10477eab/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:32.295844 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1465c01-ce45-4a53-a207-3b1868bde248 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:32.296406 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1465c01-ce45-4a53-a207-3b1868bde248 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:32.296690 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1465c01-ce45-4a53-a207-3b1868bde248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:32.302092 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:32.328039 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-f1465c01-ce45-4a53-a207-3b1868bde248 None None] MFA Rules not processed for user `3fbc4348c7c647cc864f659e10477eab`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:32.329252 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-f1465c01-ce45-4a53-a207-3b1868bde248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:32.376946 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-c72c3743-54a4-4662-9a46-45d15b33bfd4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '780716e0cd604f2aa6ee9bd8b0ca645f'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:32.385996 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-c72c3743-54a4-4662-9a46-45d15b33bfd4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '780716e0cd604f2aa6ee9bd8b0ca645f'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:32.390493 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-c72c3743-54a4-4662-9a46-45d15b33bfd4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '780716e0cd604f2aa6ee9bd8b0ca645f'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:32.413090 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-c72c3743-54a4-4662-9a46-45d15b33bfd4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '780716e0cd604f2aa6ee9bd8b0ca645f'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:32.413351 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1053/2156] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:14:32.414261 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1465c01-ce45-4a53-a207-3b1868bde248 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:32.418624 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-f1465c01-ce45-4a53-a207-3b1868bde248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:32.484307 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1465c01-ce45-4a53-a207-3b1868bde248 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:32.489238 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab63b1ea-7e3f-4dfe-976c-964e8fce72e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:32.490154 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab63b1ea-7e3f-4dfe-976c-964e8fce72e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:32.490582 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab63b1ea-7e3f-4dfe-976c-964e8fce72e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:32.490997 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab63b1ea-7e3f-4dfe-976c-964e8fce72e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:32.493928 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ab63b1ea-7e3f-4dfe-976c-964e8fce72e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:32.561832 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-c72c3743-54a4-4662-9a46-45d15b33bfd4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '780716e0cd604f2aa6ee9bd8b0ca645f'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:32.571022 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1104/2157] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:14:32 2022] DELETE /identity/v3/users/780716e0cd604f2aa6ee9bd8b0ca645f => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:14:32.575596 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c3743-54a4-4662-9a46-45d15b33bfd4 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:32.578613 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c72c3743-54a4-4662-9a46-45d15b33bfd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:32.588370 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab63b1ea-7e3f-4dfe-976c-964e8fce72e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:32.595957 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab63b1ea-7e3f-4dfe-976c-964e8fce72e0 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:32.601451 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1054/2158] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:14:32 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:14:32.611507 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee153fd-59ab-4afa-9114-05eedd46f714 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:32.612988 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee153fd-59ab-4afa-9114-05eedd46f714 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:32.613568 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee153fd-59ab-4afa-9114-05eedd46f714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:32.641084 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-1ee153fd-59ab-4afa-9114-05eedd46f714 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:32.642000 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:32.643163 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-1ee153fd-59ab-4afa-9114-05eedd46f714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:32.707493 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c3743-54a4-4662-9a46-45d15b33bfd4 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:32.712846 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e663d6-eba8-4ff4-87f7-eb841e7189c4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:32.713770 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e663d6-eba8-4ff4-87f7-eb841e7189c4 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:32.714653 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e663d6-eba8-4ff4-87f7-eb841e7189c4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:32.722208 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e663d6-eba8-4ff4-87f7-eb841e7189c4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:32.724792 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-32e663d6-eba8-4ff4-87f7-eb841e7189c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:32.760139 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1055/2159] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:32.770027 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee153fd-59ab-4afa-9114-05eedd46f714 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:32.771438 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-1ee153fd-59ab-4afa-9114-05eedd46f714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:32.823771 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e663d6-eba8-4ff4-87f7-eb841e7189c4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:32.825847 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e663d6-eba8-4ff4-87f7-eb841e7189c4 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:32.838919 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee153fd-59ab-4afa-9114-05eedd46f714 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:32.844243 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6afbc94-92e7-4c11-bf44-a7481b7ee003 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:32.844243 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afbc94-92e7-4c11-bf44-a7481b7ee003 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:32.844243 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afbc94-92e7-4c11-bf44-a7481b7ee003 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:32.844532 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1105/2160] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:14:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:14:32.844625 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6afbc94-92e7-4c11-bf44-a7481b7ee003 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:32.853928 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6afbc94-92e7-4c11-bf44-a7481b7ee003 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:32.855466 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6afbc94-92e7-4c11-bf44-a7481b7ee003 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:32.866696 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1056/2161] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:14:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 17:14:33.376383 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e663d6-eba8-4ff4-87f7-eb841e7189c4 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:33.377889 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-32e663d6-eba8-4ff4-87f7-eb841e7189c4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:33.393533 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e663d6-eba8-4ff4-87f7-eb841e7189c4 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:33.399037 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77248ad3-632a-4040-829b-ee9e5895dda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:33.399984 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77248ad3-632a-4040-829b-ee9e5895dda1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:33.400261 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77248ad3-632a-4040-829b-ee9e5895dda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:33.400737 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77248ad3-632a-4040-829b-ee9e5895dda1 admin admin] PATH_INFO: `/v3/projects/0e0b3c17322848d58f6d2984428b99ea` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:33.402306 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77248ad3-632a-4040-829b-ee9e5895dda1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e0b3c17322848d58f6d2984428b99ea)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:33.405197 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77248ad3-632a-4040-829b-ee9e5895dda1 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:33.541942 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-77248ad3-632a-4040-829b-ee9e5895dda1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e0b3c17322848d58f6d2984428b99ea', 'request_id': 'req-77248ad3-632a-4040-829b-ee9e5895dda1'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:33.549512 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-77248ad3-632a-4040-829b-ee9e5895dda1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e0b3c17322848d58f6d2984428b99ea', 'request_id': 'req-77248ad3-632a-4040-829b-ee9e5895dda1'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:33.555199 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-77248ad3-632a-4040-829b-ee9e5895dda1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e0b3c17322848d58f6d2984428b99ea', 'request_id': 'req-77248ad3-632a-4040-829b-ee9e5895dda1'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:33.624328 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-77248ad3-632a-4040-829b-ee9e5895dda1 admin admin] The token cache is being invalidate because project 0e0b3c17322848d58f6d2984428b99ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:14:33.628093 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-77248ad3-632a-4040-829b-ee9e5895dda1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:33.628093 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-77248ad3-632a-4040-829b-ee9e5895dda1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:33.628093 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1106/2162] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:14:33 2022] DELETE /identity/v3/projects/0e0b3c17322848d58f6d2984428b99ea => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:33.631091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6afbc94-92e7-4c11-bf44-a7481b7ee003 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:33.641070 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6afbc94-92e7-4c11-bf44-a7481b7ee003 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:33.699047 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e3c2b0-55cf-496d-9666-f76d6bc54dc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:33.699980 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3c2b0-55cf-496d-9666-f76d6bc54dc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:33.700264 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3c2b0-55cf-496d-9666-f76d6bc54dc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:33.700537 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e3c2b0-55cf-496d-9666-f76d6bc54dc9 admin admin] PATH_INFO: `/v3/users/8353ce80989c42fa866c64997435d6f5` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:33.701722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e3c2b0-55cf-496d-9666-f76d6bc54dc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8353ce80989c42fa866c64997435d6f5)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:33.703644 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e3c2b0-55cf-496d-9666-f76d6bc54dc9 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:33.709728 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:33.751044 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:33.921585 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-27e3c2b0-55cf-496d-9666-f76d6bc54dc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8353ce80989c42fa866c64997435d6f5'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:33.933458 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-27e3c2b0-55cf-496d-9666-f76d6bc54dc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8353ce80989c42fa866c64997435d6f5'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:33.940565 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-27e3c2b0-55cf-496d-9666-f76d6bc54dc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8353ce80989c42fa866c64997435d6f5'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:33.941279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-27e3c2b0-55cf-496d-9666-f76d6bc54dc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8353ce80989c42fa866c64997435d6f5'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:34.013819 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-27e3c2b0-55cf-496d-9666-f76d6bc54dc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8353ce80989c42fa866c64997435d6f5'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:34.024540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1057/2163] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:14:33 2022] DELETE /identity/v3/users/8353ce80989c42fa866c64997435d6f5 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:14:34.553146 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77248ad3-632a-4040-829b-ee9e5895dda1 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:34.554701 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-77248ad3-632a-4040-829b-ee9e5895dda1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:34.569057 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77248ad3-632a-4040-829b-ee9e5895dda1 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:34.629770 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db22377a-6650-4423-bcc1-1d8fe6ef9a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:34.630662 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22377a-6650-4423-bcc1-1d8fe6ef9a0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:34.630945 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22377a-6650-4423-bcc1-1d8fe6ef9a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:34.631221 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db22377a-6650-4423-bcc1-1d8fe6ef9a0c admin admin] PATH_INFO: `/v3/projects/0716872dd0514186bede1295d8c40031` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:34.632466 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db22377a-6650-4423-bcc1-1d8fe6ef9a0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0716872dd0514186bede1295d8c40031)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:34.634369 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db22377a-6650-4423-bcc1-1d8fe6ef9a0c admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:34.762386 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-db22377a-6650-4423-bcc1-1d8fe6ef9a0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0716872dd0514186bede1295d8c40031', 'request_id': 'req-db22377a-6650-4423-bcc1-1d8fe6ef9a0c'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:34.769269 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-db22377a-6650-4423-bcc1-1d8fe6ef9a0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0716872dd0514186bede1295d8c40031', 'request_id': 'req-db22377a-6650-4423-bcc1-1d8fe6ef9a0c'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:34.775514 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-db22377a-6650-4423-bcc1-1d8fe6ef9a0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0716872dd0514186bede1295d8c40031', 'request_id': 'req-db22377a-6650-4423-bcc1-1d8fe6ef9a0c'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:34.900042 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-db22377a-6650-4423-bcc1-1d8fe6ef9a0c admin admin] The token cache is being invalidate because project 0716872dd0514186bede1295d8c40031 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:14:34.900487 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-db22377a-6650-4423-bcc1-1d8fe6ef9a0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:34.901205 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-db22377a-6650-4423-bcc1-1d8fe6ef9a0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:34.903910 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1107/2164] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:14:34 2022] DELETE /identity/v3/projects/0716872dd0514186bede1295d8c40031 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:34.910078 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a22f3-1163-4b3c-ae7d-57e8bd0f0766 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:34.910078 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a22f3-1163-4b3c-ae7d-57e8bd0f0766 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:34.910667 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91a22f3-1163-4b3c-ae7d-57e8bd0f0766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:34.930717 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-d91a22f3-1163-4b3c-ae7d-57e8bd0f0766 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:34.932246 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d91a22f3-1163-4b3c-ae7d-57e8bd0f0766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:34.991860 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:35.000415 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1058/2165] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:14:35.009029 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a33b4-7a7d-4e2d-b582-87730b252382 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:35.009390 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a33b4-7a7d-4e2d-b582-87730b252382 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:35.009759 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a33b4-7a7d-4e2d-b582-87730b252382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:35.046032 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-cf7a33b4-7a7d-4e2d-b582-87730b252382 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:35.050352 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-cf7a33b4-7a7d-4e2d-b582-87730b252382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:35.056817 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1108/2166] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:14:35.060868 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a22f3-1163-4b3c-ae7d-57e8bd0f0766 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:35.072047 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91a22f3-1163-4b3c-ae7d-57e8bd0f0766 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:35.079793 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f847906-c9c2-4c55-80ae-a96c6b06ae49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:35.085073 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f847906-c9c2-4c55-80ae-a96c6b06ae49 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:35.085073 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f847906-c9c2-4c55-80ae-a96c6b06ae49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:35.085073 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f847906-c9c2-4c55-80ae-a96c6b06ae49 admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:35.085073 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f847906-c9c2-4c55-80ae-a96c6b06ae49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:14:35.085073 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f847906-c9c2-4c55-80ae-a96c6b06ae49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:35.085073 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f847906-c9c2-4c55-80ae-a96c6b06ae49 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:35.098951 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1059/2167] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:14:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:14:35.103240 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a33b4-7a7d-4e2d-b582-87730b252382 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:35.115002 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a33b4-7a7d-4e2d-b582-87730b252382 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:35.120657 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297fb526-f311-4a8d-9906-133aa9671d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:35.121575 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297fb526-f311-4a8d-9906-133aa9671d41 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:35.121866 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297fb526-f311-4a8d-9906-133aa9671d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:35.122144 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297fb526-f311-4a8d-9906-133aa9671d41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:35.122841 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297fb526-f311-4a8d-9906-133aa9671d41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:35.124927 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297fb526-f311-4a8d-9906-133aa9671d41 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:35.223646 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1109/2168] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:14:35 2022] POST /identity/v3/projects => generated 387 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:14:35.228862 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f847906-c9c2-4c55-80ae-a96c6b06ae49 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:35.240485 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f847906-c9c2-4c55-80ae-a96c6b06ae49 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:35.265195 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:35.384896 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f016d-4f55-43c0-9bbc-be5ab49bd7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:35.385694 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f016d-4f55-43c0-9bbc-be5ab49bd7ce admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:35.386055 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f016d-4f55-43c0-9bbc-be5ab49bd7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:35.386410 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f016d-4f55-43c0-9bbc-be5ab49bd7ce admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:35.387688 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f016d-4f55-43c0-9bbc-be5ab49bd7ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:35.390076 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f016d-4f55-43c0-9bbc-be5ab49bd7ce admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:35.482259 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1060/2169] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:14:35 2022] POST /identity/v3/users => generated 294 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:14:35.486550 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297fb526-f311-4a8d-9906-133aa9671d41 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:35.497373 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297fb526-f311-4a8d-9906-133aa9671d41 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:35.501870 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f0152-5258-41f8-9755-f9974aab91f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:35.502529 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f0152-5258-41f8-9755-f9974aab91f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:35.502810 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f0152-5258-41f8-9755-f9974aab91f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:35.503138 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f0152-5258-41f8-9755-f9974aab91f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:35.503880 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9f0152-5258-41f8-9755-f9974aab91f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:35.505447 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9f0152-5258-41f8-9755-f9974aab91f7 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:35.519497 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1110/2170] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:14:35.520712 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f016d-4f55-43c0-9bbc-be5ab49bd7ce admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:35.532045 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f016d-4f55-43c0-9bbc-be5ab49bd7ce admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:35.536890 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd74601-8586-43d0-95dc-7d605afe9ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:35.537558 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd74601-8586-43d0-95dc-7d605afe9ec9 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:35.537846 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd74601-8586-43d0-95dc-7d605afe9ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:35.538123 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd74601-8586-43d0-95dc-7d605afe9ec9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:35.538859 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd74601-8586-43d0-95dc-7d605afe9ec9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:35.540495 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd74601-8586-43d0-95dc-7d605afe9ec9 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:35.556349 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1061/2171] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:14:35.557238 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f0152-5258-41f8-9755-f9974aab91f7 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:35.566357 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f0152-5258-41f8-9755-f9974aab91f7 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:35.571428 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab7df9-01af-4a52-a6d0-d0d04b25005b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:35.575374 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab7df9-01af-4a52-a6d0-d0d04b25005b admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:35.575374 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab7df9-01af-4a52-a6d0-d0d04b25005b admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:35.575576 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:35.575665 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ab7df9-01af-4a52-a6d0-d0d04b25005b admin admin] PATH_INFO: `/v3/projects/c63eb983307640deae81150a415ce69f/users/138b27d3959d447eacb223bb60ac587a/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:35.588469 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ab7df9-01af-4a52-a6d0-d0d04b25005b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c63eb983307640deae81150a415ce69f, user_id=138b27d3959d447eacb223bb60ac587a, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:35.591827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ab7df9-01af-4a52-a6d0-d0d04b25005b admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:35.688323 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1111/2172] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:14:35 2022] PUT /identity/v3/projects/c63eb983307640deae81150a415ce69f/users/138b27d3959d447eacb223bb60ac587a/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:35.699340 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c2d9a-dccb-4932-9943-7bc1211e3e93 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:35.700190 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c2d9a-dccb-4932-9943-7bc1211e3e93 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:35.700460 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c2d9a-dccb-4932-9943-7bc1211e3e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:35.728716 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-672c2d9a-dccb-4932-9943-7bc1211e3e93 None None] MFA Rules not processed for user `138b27d3959d447eacb223bb60ac587a`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:35.729837 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-672c2d9a-dccb-4932-9943-7bc1211e3e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:35.785944 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:35.798031 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1062/2173] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:35.802670 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab7df9-01af-4a52-a6d0-d0d04b25005b admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:35.865635 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ab7df9-01af-4a52-a6d0-d0d04b25005b admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:35.870307 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957ec32-40bc-4bb0-961a-df5b62de5aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:35.871004 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957ec32-40bc-4bb0-961a-df5b62de5aef admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:35.871865 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957ec32-40bc-4bb0-961a-df5b62de5aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:35.872149 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957ec32-40bc-4bb0-961a-df5b62de5aef admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:35.872842 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d957ec32-40bc-4bb0-961a-df5b62de5aef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:35.874755 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d957ec32-40bc-4bb0-961a-df5b62de5aef admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:35.969061 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1112/2174] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:14:35 2022] POST /identity/v3/projects => generated 387 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:14:35.973558 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672c2d9a-dccb-4932-9943-7bc1211e3e93 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:35.984311 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672c2d9a-dccb-4932-9943-7bc1211e3e93 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:36.048510 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fceea1-bb8c-4ed5-a89e-aa6b6e56fc63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:36.049190 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fceea1-bb8c-4ed5-a89e-aa6b6e56fc63 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:36.049750 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fceea1-bb8c-4ed5-a89e-aa6b6e56fc63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:36.050053 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fceea1-bb8c-4ed5-a89e-aa6b6e56fc63 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:36.050759 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fceea1-bb8c-4ed5-a89e-aa6b6e56fc63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:36.052742 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fceea1-bb8c-4ed5-a89e-aa6b6e56fc63 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:36.082040 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:36.170375 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1063/2175] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:14:35 2022] POST /identity/v3/users => generated 300 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:14:36.171854 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957ec32-40bc-4bb0-961a-df5b62de5aef admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:36.182492 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957ec32-40bc-4bb0-961a-df5b62de5aef admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:36.187934 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831412d8-7a31-4562-a58b-1236f32ba3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:36.188628 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831412d8-7a31-4562-a58b-1236f32ba3c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:36.189024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831412d8-7a31-4562-a58b-1236f32ba3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:36.189396 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831412d8-7a31-4562-a58b-1236f32ba3c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:36.190501 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831412d8-7a31-4562-a58b-1236f32ba3c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:36.192185 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831412d8-7a31-4562-a58b-1236f32ba3c1 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:36.209680 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1113/2176] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:14:36.215132 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fceea1-bb8c-4ed5-a89e-aa6b6e56fc63 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:36.225788 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fceea1-bb8c-4ed5-a89e-aa6b6e56fc63 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:36.240818 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c23ed74-eae9-4d9e-8897-16fd3f8b4a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:36.240818 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23ed74-eae9-4d9e-8897-16fd3f8b4a6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:36.240818 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23ed74-eae9-4d9e-8897-16fd3f8b4a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:36.240818 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23ed74-eae9-4d9e-8897-16fd3f8b4a6b admin admin] PATH_INFO: `/v3/projects/816b199982c94adbb8702b58ad06d4c7/users/ae83b0aeef624331a3de54f05e796ed4/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:36.256812 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c23ed74-eae9-4d9e-8897-16fd3f8b4a6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=816b199982c94adbb8702b58ad06d4c7, user_id=ae83b0aeef624331a3de54f05e796ed4, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:36.259548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c23ed74-eae9-4d9e-8897-16fd3f8b4a6b admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:36.265233 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:36.299573 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:36.437428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1064/2177] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:14:36 2022] PUT /identity/v3/projects/816b199982c94adbb8702b58ad06d4c7/users/ae83b0aeef624331a3de54f05e796ed4/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:14:36.441823 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831412d8-7a31-4562-a58b-1236f32ba3c1 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:36.452146 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831412d8-7a31-4562-a58b-1236f32ba3c1 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:36.518974 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32d22e8-9998-4dd2-850d-c5f08199ce2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:36.519887 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d22e8-9998-4dd2-850d-c5f08199ce2c admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:36.520182 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d22e8-9998-4dd2-850d-c5f08199ce2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:36.524859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32d22e8-9998-4dd2-850d-c5f08199ce2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:36.524859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32d22e8-9998-4dd2-850d-c5f08199ce2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:36.524859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32d22e8-9998-4dd2-850d-c5f08199ce2c admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:36.533962 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1114/2178] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:36 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:14:36.538378 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c23ed74-eae9-4d9e-8897-16fd3f8b4a6b admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:36.549673 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c23ed74-eae9-4d9e-8897-16fd3f8b4a6b admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:36.554279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a9786-58ce-450f-955f-d6e135ecec05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:36.555147 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a9786-58ce-450f-955f-d6e135ecec05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:36.555428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a9786-58ce-450f-955f-d6e135ecec05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:36.555702 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816a9786-58ce-450f-955f-d6e135ecec05 admin admin] PATH_INFO: `/v3/projects/816b199982c94adbb8702b58ad06d4c7/users/ae83b0aeef624331a3de54f05e796ed4/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:36.557670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816a9786-58ce-450f-955f-d6e135ecec05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=816b199982c94adbb8702b58ad06d4c7, user_id=ae83b0aeef624331a3de54f05e796ed4, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:36.560362 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816a9786-58ce-450f-955f-d6e135ecec05 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:36.570172 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1065/2179] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:14:36 2022] PUT /identity/v3/projects/816b199982c94adbb8702b58ad06d4c7/users/ae83b0aeef624331a3de54f05e796ed4/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:14:36.581101 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51936367-0ad9-45f0-a39d-4d1fe6e88af5 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:36.582715 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51936367-0ad9-45f0-a39d-4d1fe6e88af5 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:36.582715 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51936367-0ad9-45f0-a39d-4d1fe6e88af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:36.608932 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-51936367-0ad9-45f0-a39d-4d1fe6e88af5 None None] MFA Rules not processed for user `ae83b0aeef624331a3de54f05e796ed4`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:36.610151 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-51936367-0ad9-45f0-a39d-4d1fe6e88af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:36.627869 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:36.681157 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1115/2180] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:36 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:14:36.692729 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a9786-58ce-450f-955f-d6e135ecec05 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:36.694975 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-816a9786-58ce-450f-955f-d6e135ecec05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:36.760888 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816a9786-58ce-450f-955f-d6e135ecec05 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:36.765054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5f6bc5-013b-4e46-9d7d-60eba293916a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:36.766535 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f6bc5-013b-4e46-9d7d-60eba293916a service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:36.766830 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f6bc5-013b-4e46-9d7d-60eba293916a service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:36.767158 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5f6bc5-013b-4e46-9d7d-60eba293916a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:36.777014 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5f6bc5-013b-4e46-9d7d-60eba293916a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:36.779084 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5f6bc5-013b-4e46-9d7d-60eba293916a service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:36.788707 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1066/2181] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:14:36 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:14:36.815858 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51936367-0ad9-45f0-a39d-4d1fe6e88af5 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:36.817999 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:36.879484 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51936367-0ad9-45f0-a39d-4d1fe6e88af5 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:36.884101 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45974a3-2646-444f-a88c-7b4611efec98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:36.884961 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45974a3-2646-444f-a88c-7b4611efec98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:36.885244 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45974a3-2646-444f-a88c-7b4611efec98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:36.885519 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45974a3-2646-444f-a88c-7b4611efec98 admin admin] PATH_INFO: `/v3/users/138b27d3959d447eacb223bb60ac587a` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:36.887270 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45974a3-2646-444f-a88c-7b4611efec98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=138b27d3959d447eacb223bb60ac587a)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:36.889253 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45974a3-2646-444f-a88c-7b4611efec98 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:36.932970 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:37.048290 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f45974a3-2646-444f-a88c-7b4611efec98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '138b27d3959d447eacb223bb60ac587a'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:37.054976 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f45974a3-2646-444f-a88c-7b4611efec98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '138b27d3959d447eacb223bb60ac587a'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:37.055683 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f45974a3-2646-444f-a88c-7b4611efec98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '138b27d3959d447eacb223bb60ac587a'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:37.067819 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f45974a3-2646-444f-a88c-7b4611efec98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '138b27d3959d447eacb223bb60ac587a'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:37.110604 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:37.127088 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f45974a3-2646-444f-a88c-7b4611efec98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '138b27d3959d447eacb223bb60ac587a'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:37.133271 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1116/2182] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:14:36 2022] DELETE /identity/v3/users/138b27d3959d447eacb223bb60ac587a => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:37.138311 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da01a75-e904-42be-aa7b-b1cb2daafce0 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:37.138599 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da01a75-e904-42be-aa7b-b1cb2daafce0 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:37.138882 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da01a75-e904-42be-aa7b-b1cb2daafce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:37.156454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-2da01a75-e904-42be-aa7b-b1cb2daafce0 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:37.157655 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-2da01a75-e904-42be-aa7b-b1cb2daafce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:37.190524 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:37.208959 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:37.217126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1067/2183] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:37.226724 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45974a3-2646-444f-a88c-7b4611efec98 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:37.228214 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f45974a3-2646-444f-a88c-7b4611efec98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:37.282151 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:37.284341 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45974a3-2646-444f-a88c-7b4611efec98 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:37.288069 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140c1f93-6470-41b6-80c4-d6a77b1c9157 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:37.288714 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c1f93-6470-41b6-80c4-d6a77b1c9157 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:37.298956 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c1f93-6470-41b6-80c4-d6a77b1c9157 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:37.298956 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140c1f93-6470-41b6-80c4-d6a77b1c9157 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:37.298956 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140c1f93-6470-41b6-80c4-d6a77b1c9157 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:37.300003 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140c1f93-6470-41b6-80c4-d6a77b1c9157 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:37.309955 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1117/2184] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:14:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 17:14:37.316265 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:37.586291 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:37.704829 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:37.769075 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da01a75-e904-42be-aa7b-b1cb2daafce0 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:37.771594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-2da01a75-e904-42be-aa7b-b1cb2daafce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:37.784757 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da01a75-e904-42be-aa7b-b1cb2daafce0 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:37.789750 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280e5d6-d498-4722-abe3-b5ce5f993963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:37.790960 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280e5d6-d498-4722-abe3-b5ce5f993963 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:37.791265 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280e5d6-d498-4722-abe3-b5ce5f993963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:37.791597 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8280e5d6-d498-4722-abe3-b5ce5f993963 admin admin] PATH_INFO: `/v3/projects/c63eb983307640deae81150a415ce69f` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:37.793773 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8280e5d6-d498-4722-abe3-b5ce5f993963 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c63eb983307640deae81150a415ce69f)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:37.798075 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8280e5d6-d498-4722-abe3-b5ce5f993963 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:37.923309 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-8280e5d6-d498-4722-abe3-b5ce5f993963 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c63eb983307640deae81150a415ce69f', 'request_id': 'req-8280e5d6-d498-4722-abe3-b5ce5f993963'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:37.930015 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-8280e5d6-d498-4722-abe3-b5ce5f993963 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c63eb983307640deae81150a415ce69f', 'request_id': 'req-8280e5d6-d498-4722-abe3-b5ce5f993963'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:37.996067 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-8280e5d6-d498-4722-abe3-b5ce5f993963 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c63eb983307640deae81150a415ce69f', 'request_id': 'req-8280e5d6-d498-4722-abe3-b5ce5f993963'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:38.007642 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-8280e5d6-d498-4722-abe3-b5ce5f993963 admin admin] The token cache is being invalidate because project c63eb983307640deae81150a415ce69f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:14:38.007962 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-8280e5d6-d498-4722-abe3-b5ce5f993963 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:38.008595 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-8280e5d6-d498-4722-abe3-b5ce5f993963 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:38.010133 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1068/2185] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:14:37 2022] DELETE /identity/v3/projects/c63eb983307640deae81150a415ce69f => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:38.014739 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140c1f93-6470-41b6-80c4-d6a77b1c9157 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:38.024657 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140c1f93-6470-41b6-80c4-d6a77b1c9157 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:38.084696 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa0cfa1-9d8a-428b-b9b6-4c36560f3bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:38.085568 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa0cfa1-9d8a-428b-b9b6-4c36560f3bef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:38.085848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa0cfa1-9d8a-428b-b9b6-4c36560f3bef admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:38.086124 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa0cfa1-9d8a-428b-b9b6-4c36560f3bef admin admin] PATH_INFO: `/v3/users/ae83b0aeef624331a3de54f05e796ed4` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:38.087457 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa0cfa1-9d8a-428b-b9b6-4c36560f3bef admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae83b0aeef624331a3de54f05e796ed4)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:38.089555 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa0cfa1-9d8a-428b-b9b6-4c36560f3bef admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:38.102626 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:38.210275 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1aa0cfa1-9d8a-428b-b9b6-4c36560f3bef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae83b0aeef624331a3de54f05e796ed4'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:38.218409 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1aa0cfa1-9d8a-428b-b9b6-4c36560f3bef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae83b0aeef624331a3de54f05e796ed4'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:38.219102 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1aa0cfa1-9d8a-428b-b9b6-4c36560f3bef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae83b0aeef624331a3de54f05e796ed4'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:38.231235 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1aa0cfa1-9d8a-428b-b9b6-4c36560f3bef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae83b0aeef624331a3de54f05e796ed4'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:38.339067 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1aa0cfa1-9d8a-428b-b9b6-4c36560f3bef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae83b0aeef624331a3de54f05e796ed4'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:38.346527 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1118/2186] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:14:38 2022] DELETE /identity/v3/users/ae83b0aeef624331a3de54f05e796ed4 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:38.723794 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280e5d6-d498-4722-abe3-b5ce5f993963 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:38.725464 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-8280e5d6-d498-4722-abe3-b5ce5f993963 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:38.739660 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8280e5d6-d498-4722-abe3-b5ce5f993963 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:38.806135 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570dfe9e-d29d-4d14-ba7a-f1c8bd7e265f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:38.807269 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570dfe9e-d29d-4d14-ba7a-f1c8bd7e265f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:38.807269 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570dfe9e-d29d-4d14-ba7a-f1c8bd7e265f admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:38.807538 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570dfe9e-d29d-4d14-ba7a-f1c8bd7e265f admin admin] PATH_INFO: `/v3/projects/816b199982c94adbb8702b58ad06d4c7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:38.808759 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570dfe9e-d29d-4d14-ba7a-f1c8bd7e265f admin admin] RBAC: Authorizing `identity:delete_project(project_id=816b199982c94adbb8702b58ad06d4c7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:38.811556 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570dfe9e-d29d-4d14-ba7a-f1c8bd7e265f admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:38.944617 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:38.955134 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-570dfe9e-d29d-4d14-ba7a-f1c8bd7e265f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '816b199982c94adbb8702b58ad06d4c7', 'request_id': 'req-570dfe9e-d29d-4d14-ba7a-f1c8bd7e265f'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:38.963844 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-570dfe9e-d29d-4d14-ba7a-f1c8bd7e265f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '816b199982c94adbb8702b58ad06d4c7', 'request_id': 'req-570dfe9e-d29d-4d14-ba7a-f1c8bd7e265f'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:39.047511 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-570dfe9e-d29d-4d14-ba7a-f1c8bd7e265f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '816b199982c94adbb8702b58ad06d4c7', 'request_id': 'req-570dfe9e-d29d-4d14-ba7a-f1c8bd7e265f'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:39.059512 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-570dfe9e-d29d-4d14-ba7a-f1c8bd7e265f admin admin] The token cache is being invalidate because project 816b199982c94adbb8702b58ad06d4c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:14:39.059828 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-570dfe9e-d29d-4d14-ba7a-f1c8bd7e265f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:39.061163 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-570dfe9e-d29d-4d14-ba7a-f1c8bd7e265f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:39.062783 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1069/2187] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:14:38 2022] DELETE /identity/v3/projects/816b199982c94adbb8702b58ad06d4c7 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:39.068133 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c27763-9370-4e06-8fe2-e63dbc0b55d8 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:39.068619 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c27763-9370-4e06-8fe2-e63dbc0b55d8 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:39.071782 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c27763-9370-4e06-8fe2-e63dbc0b55d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:39.088147 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-94c27763-9370-4e06-8fe2-e63dbc0b55d8 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:39.089384 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-94c27763-9370-4e06-8fe2-e63dbc0b55d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:39.149762 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1119/2188] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:39.157809 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a0068-54af-4e69-85b6-b72de38013f9 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:39.158098 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a0068-54af-4e69-85b6-b72de38013f9 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:39.158435 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635a0068-54af-4e69-85b6-b72de38013f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:39.178336 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-635a0068-54af-4e69-85b6-b72de38013f9 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:39.179649 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-635a0068-54af-4e69-85b6-b72de38013f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:39.188884 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1070/2189] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:39.193649 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c27763-9370-4e06-8fe2-e63dbc0b55d8 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:39.203555 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c27763-9370-4e06-8fe2-e63dbc0b55d8 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:39.208231 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bbd5b5-5da6-4369-8530-a5b2657cf07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:39.208887 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bbd5b5-5da6-4369-8530-a5b2657cf07b admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:39.209170 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bbd5b5-5da6-4369-8530-a5b2657cf07b admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:39.209449 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bbd5b5-5da6-4369-8530-a5b2657cf07b admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:39.210172 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bbd5b5-5da6-4369-8530-a5b2657cf07b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:14:39.213841 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bbd5b5-5da6-4369-8530-a5b2657cf07b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:39.213841 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bbd5b5-5da6-4369-8530-a5b2657cf07b admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:39.227491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1120/2190] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:14:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:14:39.228993 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635a0068-54af-4e69-85b6-b72de38013f9 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:39.238662 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635a0068-54af-4e69-85b6-b72de38013f9 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:39.243444 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e33c98-e52e-4215-a4c8-707b69be7819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:39.244194 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e33c98-e52e-4215-a4c8-707b69be7819 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:39.244545 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e33c98-e52e-4215-a4c8-707b69be7819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:39.244828 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e33c98-e52e-4215-a4c8-707b69be7819 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:39.245675 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e33c98-e52e-4215-a4c8-707b69be7819 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:39.248026 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e33c98-e52e-4215-a4c8-707b69be7819 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:39.361038 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1071/2191] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:14:39 2022] POST /identity/v3/projects => generated 393 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:14:39.363760 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bbd5b5-5da6-4369-8530-a5b2657cf07b admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:39.374717 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bbd5b5-5da6-4369-8530-a5b2657cf07b admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:39.439269 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc77495-fad1-4ab4-81e2-2b419a44a5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:39.439980 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc77495-fad1-4ab4-81e2-2b419a44a5f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:39.440271 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc77495-fad1-4ab4-81e2-2b419a44a5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:39.440546 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc77495-fad1-4ab4-81e2-2b419a44a5f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:39.441275 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc77495-fad1-4ab4-81e2-2b419a44a5f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:39.443285 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc77495-fad1-4ab4-81e2-2b419a44a5f5 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:39.540304 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1121/2192] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:14:39 2022] POST /identity/v3/users => generated 297 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:14:39.546405 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e33c98-e52e-4215-a4c8-707b69be7819 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:39.556795 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e33c98-e52e-4215-a4c8-707b69be7819 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:39.561580 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff1d70-fb27-4359-9050-10aca34ba1a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:39.562241 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff1d70-fb27-4359-9050-10aca34ba1a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:39.562524 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff1d70-fb27-4359-9050-10aca34ba1a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:39.562798 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ff1d70-fb27-4359-9050-10aca34ba1a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:39.563527 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ff1d70-fb27-4359-9050-10aca34ba1a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:39.565361 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ff1d70-fb27-4359-9050-10aca34ba1a5 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:39.577373 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1072/2193] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:14:39.583100 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc77495-fad1-4ab4-81e2-2b419a44a5f5 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:39.593958 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc77495-fad1-4ab4-81e2-2b419a44a5f5 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:39.598630 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0c4e93-94c8-4537-a946-d4c06dd95b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:39.599315 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c4e93-94c8-4537-a946-d4c06dd95b88 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:39.599605 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c4e93-94c8-4537-a946-d4c06dd95b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:39.599926 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0c4e93-94c8-4537-a946-d4c06dd95b88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:39.600662 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0c4e93-94c8-4537-a946-d4c06dd95b88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:39.602256 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0c4e93-94c8-4537-a946-d4c06dd95b88 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:39.603568 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:39.617479 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1122/2194] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:14:39.618898 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff1d70-fb27-4359-9050-10aca34ba1a5 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:39.633046 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ff1d70-fb27-4359-9050-10aca34ba1a5 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:39.645938 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bff6c3-6235-4ab5-8d37-b3dc26dbb356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:39.645938 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bff6c3-6235-4ab5-8d37-b3dc26dbb356 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:39.645938 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bff6c3-6235-4ab5-8d37-b3dc26dbb356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:39.645938 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bff6c3-6235-4ab5-8d37-b3dc26dbb356 admin admin] PATH_INFO: `/v3/projects/e17a8cb8565d4a80be3a244fa6d71dad/users/2d6e29ab397e4f51a4695069895395dc/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:39.653626 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bff6c3-6235-4ab5-8d37-b3dc26dbb356 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e17a8cb8565d4a80be3a244fa6d71dad, user_id=2d6e29ab397e4f51a4695069895395dc, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:39.656851 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bff6c3-6235-4ab5-8d37-b3dc26dbb356 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:39.730003 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1073/2195] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:14:39 2022] PUT /identity/v3/projects/e17a8cb8565d4a80be3a244fa6d71dad/users/2d6e29ab397e4f51a4695069895395dc/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:39.742537 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6b792-5323-41a7-b653-cd76b25c073c None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:39.743012 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6b792-5323-41a7-b653-cd76b25c073c None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:39.743290 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e6b792-5323-41a7-b653-cd76b25c073c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:39.774089 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-f7e6b792-5323-41a7-b653-cd76b25c073c None None] MFA Rules not processed for user `2d6e29ab397e4f51a4695069895395dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:39.775306 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f7e6b792-5323-41a7-b653-cd76b25c073c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:39.843876 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1123/2196] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:39 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:39.848447 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bff6c3-6235-4ab5-8d37-b3dc26dbb356 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:39.927253 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bff6c3-6235-4ab5-8d37-b3dc26dbb356 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:39.932665 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b56656-537c-46e3-b47c-708cf1346833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:39.933349 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b56656-537c-46e3-b47c-708cf1346833 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:39.933703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b56656-537c-46e3-b47c-708cf1346833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:39.933895 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b56656-537c-46e3-b47c-708cf1346833 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:39.934676 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b56656-537c-46e3-b47c-708cf1346833 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:39.936729 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b56656-537c-46e3-b47c-708cf1346833 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:39.956720 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e6b792-5323-41a7-b653-cd76b25c073c None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:39.957522 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f7e6b792-5323-41a7-b653-cd76b25c073c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:39.972346 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e6b792-5323-41a7-b653-cd76b25c073c None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:39.977170 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49a9a31-87ae-4dab-89d8-46e22523a047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:39.978115 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a9a31-87ae-4dab-89d8-46e22523a047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:39.981945 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a9a31-87ae-4dab-89d8-46e22523a047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:39.981945 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49a9a31-87ae-4dab-89d8-46e22523a047 admin admin] PATH_INFO: `/v3/users/6fb86720d1cd4a70bec442ddbfb5c6a0` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:39.981945 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49a9a31-87ae-4dab-89d8-46e22523a047 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fb86720d1cd4a70bec442ddbfb5c6a0)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:39.982263 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49a9a31-87ae-4dab-89d8-46e22523a047 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:40.043299 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1074/2197] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:14:39 2022] POST /identity/v3/projects => generated 393 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:14:40.044598 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b56656-537c-46e3-b47c-708cf1346833 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:40.099763 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:40.112276 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b56656-537c-46e3-b47c-708cf1346833 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:40.116928 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c64feb1-4669-42a1-bb12-43a2ac798111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:40.117597 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64feb1-4669-42a1-bb12-43a2ac798111 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:40.117884 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64feb1-4669-42a1-bb12-43a2ac798111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:40.118162 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c64feb1-4669-42a1-bb12-43a2ac798111 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:40.118848 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c64feb1-4669-42a1-bb12-43a2ac798111 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:40.120836 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c64feb1-4669-42a1-bb12-43a2ac798111 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:40.179909 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-a49a9a31-87ae-4dab-89d8-46e22523a047 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fb86720d1cd4a70bec442ddbfb5c6a0'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:40.188344 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-a49a9a31-87ae-4dab-89d8-46e22523a047 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fb86720d1cd4a70bec442ddbfb5c6a0'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:40.189041 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-a49a9a31-87ae-4dab-89d8-46e22523a047 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fb86720d1cd4a70bec442ddbfb5c6a0'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:40.204541 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-a49a9a31-87ae-4dab-89d8-46e22523a047 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fb86720d1cd4a70bec442ddbfb5c6a0'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:40.246427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1075/2198] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:14:40 2022] POST /identity/v3/users => generated 303 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:14:40.250900 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c64feb1-4669-42a1-bb12-43a2ac798111 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:40.260786 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c64feb1-4669-42a1-bb12-43a2ac798111 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:40.263808 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-a49a9a31-87ae-4dab-89d8-46e22523a047 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fb86720d1cd4a70bec442ddbfb5c6a0'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:40.269354 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be211d-5597-4ae8-b4a6-c573b0ab268e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:40.272715 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be211d-5597-4ae8-b4a6-c573b0ab268e admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:40.272715 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be211d-5597-4ae8-b4a6-c573b0ab268e admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:40.272715 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76be211d-5597-4ae8-b4a6-c573b0ab268e admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:40.272955 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1124/2199] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:14:39 2022] DELETE /identity/v3/users/6fb86720d1cd4a70bec442ddbfb5c6a0 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:40.274402 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76be211d-5597-4ae8-b4a6-c573b0ab268e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:40.275947 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72facefe-5bfb-405b-8bf7-21e901e8404b None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:40.276148 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76be211d-5597-4ae8-b4a6-c573b0ab268e admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:40.276995 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72facefe-5bfb-405b-8bf7-21e901e8404b None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:40.277458 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72facefe-5bfb-405b-8bf7-21e901e8404b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:40.295260 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1076/2200] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:14:40.296576 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be211d-5597-4ae8-b4a6-c573b0ab268e admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:40.301297 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-72facefe-5bfb-405b-8bf7-21e901e8404b None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:40.301297 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-72facefe-5bfb-405b-8bf7-21e901e8404b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:40.301464 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-76be211d-5597-4ae8-b4a6-c573b0ab268e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:40.313216 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76be211d-5597-4ae8-b4a6-c573b0ab268e admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:40.362827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:40.374727 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1125/2201] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:14:40.384875 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72facefe-5bfb-405b-8bf7-21e901e8404b None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:40.386411 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-72facefe-5bfb-405b-8bf7-21e901e8404b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:40.390020 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd085fa-a827-48ad-bf68-1aff74ed1ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:40.390904 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd085fa-a827-48ad-bf68-1aff74ed1ce7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:40.391183 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd085fa-a827-48ad-bf68-1aff74ed1ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:40.391455 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd085fa-a827-48ad-bf68-1aff74ed1ce7 admin admin] PATH_INFO: `/v3/projects/6d366bf0aae84c95adfff19b95045648/users/d52ffade10eb49dba76006657964164d/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:40.409363 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd085fa-a827-48ad-bf68-1aff74ed1ce7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d366bf0aae84c95adfff19b95045648, user_id=d52ffade10eb49dba76006657964164d, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:40.412136 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd085fa-a827-48ad-bf68-1aff74ed1ce7 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:40.453754 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72facefe-5bfb-405b-8bf7-21e901e8404b None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:40.457616 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b128f2-d6fd-4027-ad99-9501f103762e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:40.458265 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b128f2-d6fd-4027-ad99-9501f103762e service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:40.458547 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b128f2-d6fd-4027-ad99-9501f103762e service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:40.458822 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b128f2-d6fd-4027-ad99-9501f103762e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:40.469290 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b128f2-d6fd-4027-ad99-9501f103762e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:40.470841 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b128f2-d6fd-4027-ad99-9501f103762e service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:40.484351 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1077/2202] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:14:40 2022] PUT /identity/v3/projects/6d366bf0aae84c95adfff19b95045648/users/d52ffade10eb49dba76006657964164d/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:40.489359 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1126/2203] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:14:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 17:14:40.489534 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd085fa-a827-48ad-bf68-1aff74ed1ce7 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:40.570655 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd085fa-a827-48ad-bf68-1aff74ed1ce7 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:40.576419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4e4982-c978-4069-b518-9355fdb16ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:40.577150 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e4982-c978-4069-b518-9355fdb16ca1 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:40.577435 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e4982-c978-4069-b518-9355fdb16ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:40.577716 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e4982-c978-4069-b518-9355fdb16ca1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:40.578510 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4e4982-c978-4069-b518-9355fdb16ca1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:40.580172 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4e4982-c978-4069-b518-9355fdb16ca1 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:40.594303 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1078/2204] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:40 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:14:40.595818 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b128f2-d6fd-4027-ad99-9501f103762e service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:40.609282 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b128f2-d6fd-4027-ad99-9501f103762e service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:40.616434 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ee06d-7872-43d1-95ab-3d841ea8dcb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:40.617480 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ee06d-7872-43d1-95ab-3d841ea8dcb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:40.621690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ee06d-7872-43d1-95ab-3d841ea8dcb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:40.621690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507ee06d-7872-43d1-95ab-3d841ea8dcb5 admin admin] PATH_INFO: `/v3/projects/6d366bf0aae84c95adfff19b95045648/users/d52ffade10eb49dba76006657964164d/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:40.626350 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507ee06d-7872-43d1-95ab-3d841ea8dcb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d366bf0aae84c95adfff19b95045648, user_id=d52ffade10eb49dba76006657964164d, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:40.627005 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507ee06d-7872-43d1-95ab-3d841ea8dcb5 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:40.638538 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1127/2205] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:14:40 2022] PUT /identity/v3/projects/6d366bf0aae84c95adfff19b95045648/users/d52ffade10eb49dba76006657964164d/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:40.650245 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e0c60-53ec-49e2-9499-d9f9781113e1 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:40.650543 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e0c60-53ec-49e2-9499-d9f9781113e1 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:40.650824 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924e0c60-53ec-49e2-9499-d9f9781113e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:40.694940 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-924e0c60-53ec-49e2-9499-d9f9781113e1 None None] MFA Rules not processed for user `d52ffade10eb49dba76006657964164d`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:40.696971 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-924e0c60-53ec-49e2-9499-d9f9781113e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:40.744003 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:40.791364 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1079/2206] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:40 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:14:40.801403 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ee06d-7872-43d1-95ab-3d841ea8dcb5 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:40.804961 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-507ee06d-7872-43d1-95ab-3d841ea8dcb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:40.874616 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507ee06d-7872-43d1-95ab-3d841ea8dcb5 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:40.878558 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156223bd-0819-4b56-acb1-ed7fd468496c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:40.879341 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156223bd-0819-4b56-acb1-ed7fd468496c service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:40.879694 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156223bd-0819-4b56-acb1-ed7fd468496c service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:40.880080 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156223bd-0819-4b56-acb1-ed7fd468496c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:40.890470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156223bd-0819-4b56-acb1-ed7fd468496c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:40.892030 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156223bd-0819-4b56-acb1-ed7fd468496c service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:40.902122 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1128/2207] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:14:40 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 17:14:40.924778 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924e0c60-53ec-49e2-9499-d9f9781113e1 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:40.926545 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-924e0c60-53ec-49e2-9499-d9f9781113e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:40.966273 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156223bd-0819-4b56-acb1-ed7fd468496c service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:40.968662 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-156223bd-0819-4b56-acb1-ed7fd468496c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:40.983625 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156223bd-0819-4b56-acb1-ed7fd468496c service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:40.987540 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a04bfd-6585-4a5b-84d9-1ec25b04a475 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:40.988250 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a04bfd-6585-4a5b-84d9-1ec25b04a475 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:40.988518 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a04bfd-6585-4a5b-84d9-1ec25b04a475 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:40.988809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a04bfd-6585-4a5b-84d9-1ec25b04a475 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:40.990647 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d5a04bfd-6585-4a5b-84d9-1ec25b04a475 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:41.015962 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924e0c60-53ec-49e2-9499-d9f9781113e1 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:41.021665 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6c51b1-1cb3-4a0b-b82e-f58d658e2acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:41.023481 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6c51b1-1cb3-4a0b-b82e-f58d658e2acb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:41.023481 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6c51b1-1cb3-4a0b-b82e-f58d658e2acb admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:41.023481 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6c51b1-1cb3-4a0b-b82e-f58d658e2acb admin admin] PATH_INFO: `/v3/projects/4bee457a673b424f8fabdcb88c4f1d4d` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:41.024525 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6c51b1-1cb3-4a0b-b82e-f58d658e2acb admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bee457a673b424f8fabdcb88c4f1d4d)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:41.028294 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6c51b1-1cb3-4a0b-b82e-f58d658e2acb admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:41.067993 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a04bfd-6585-4a5b-84d9-1ec25b04a475 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:41.069461 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a04bfd-6585-4a5b-84d9-1ec25b04a475 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:41.079705 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1129/2208] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:14:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:14:41.139023 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ad6c51b1-1cb3-4a0b-b82e-f58d658e2acb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bee457a673b424f8fabdcb88c4f1d4d', 'request_id': 'req-ad6c51b1-1cb3-4a0b-b82e-f58d658e2acb'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:41.147388 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ad6c51b1-1cb3-4a0b-b82e-f58d658e2acb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bee457a673b424f8fabdcb88c4f1d4d', 'request_id': 'req-ad6c51b1-1cb3-4a0b-b82e-f58d658e2acb'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:41.208615 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ad6c51b1-1cb3-4a0b-b82e-f58d658e2acb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bee457a673b424f8fabdcb88c4f1d4d', 'request_id': 'req-ad6c51b1-1cb3-4a0b-b82e-f58d658e2acb'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:41.225737 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ad6c51b1-1cb3-4a0b-b82e-f58d658e2acb admin admin] The token cache is being invalidate because project 4bee457a673b424f8fabdcb88c4f1d4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:14:41.226040 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ad6c51b1-1cb3-4a0b-b82e-f58d658e2acb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:41.226756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ad6c51b1-1cb3-4a0b-b82e-f58d658e2acb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:41.228995 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1080/2209] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:14:40 2022] DELETE /identity/v3/projects/4bee457a673b424f8fabdcb88c4f1d4d => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:41.233228 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a04bfd-6585-4a5b-84d9-1ec25b04a475 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:41.248873 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a04bfd-6585-4a5b-84d9-1ec25b04a475 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:41.278110 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:41.314362 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b7eca7-d353-46ca-b186-8dfff2edb35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:41.315245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b7eca7-d353-46ca-b186-8dfff2edb35c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:41.315529 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b7eca7-d353-46ca-b186-8dfff2edb35c admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:41.315838 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b7eca7-d353-46ca-b186-8dfff2edb35c admin admin] PATH_INFO: `/v3/users/bf9b62c9870d435f90d1ef177d6d0dc7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:41.317059 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b7eca7-d353-46ca-b186-8dfff2edb35c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf9b62c9870d435f90d1ef177d6d0dc7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:41.319014 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b7eca7-d353-46ca-b186-8dfff2edb35c admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:41.635097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-89b7eca7-d353-46ca-b186-8dfff2edb35c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf9b62c9870d435f90d1ef177d6d0dc7'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:41.643306 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-89b7eca7-d353-46ca-b186-8dfff2edb35c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf9b62c9870d435f90d1ef177d6d0dc7'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:41.644081 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-89b7eca7-d353-46ca-b186-8dfff2edb35c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf9b62c9870d435f90d1ef177d6d0dc7'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:41.658890 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-89b7eca7-d353-46ca-b186-8dfff2edb35c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf9b62c9870d435f90d1ef177d6d0dc7'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:41.744527 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-89b7eca7-d353-46ca-b186-8dfff2edb35c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf9b62c9870d435f90d1ef177d6d0dc7'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:41.752267 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1130/2210] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:14:41 2022] DELETE /identity/v3/users/bf9b62c9870d435f90d1ef177d6d0dc7 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:14:42.239802 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6c51b1-1cb3-4a0b-b82e-f58d658e2acb admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:42.245972 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ad6c51b1-1cb3-4a0b-b82e-f58d658e2acb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:42.283479 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6c51b1-1cb3-4a0b-b82e-f58d658e2acb admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:42.395628 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1a7b5a-3f55-4d66-ba1e-711f2ecec74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:42.397283 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a7b5a-3f55-4d66-ba1e-711f2ecec74b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:42.397283 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a7b5a-3f55-4d66-ba1e-711f2ecec74b admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:42.397467 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a7b5a-3f55-4d66-ba1e-711f2ecec74b admin admin] PATH_INFO: `/v3/projects/8c34969de69a4287a65ff0f4fe026b75` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:42.399276 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1a7b5a-3f55-4d66-ba1e-711f2ecec74b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c34969de69a4287a65ff0f4fe026b75)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:42.402657 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1a7b5a-3f55-4d66-ba1e-711f2ecec74b admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:42.405275 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:42.534512 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-1d1a7b5a-3f55-4d66-ba1e-711f2ecec74b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c34969de69a4287a65ff0f4fe026b75', 'request_id': 'req-1d1a7b5a-3f55-4d66-ba1e-711f2ecec74b'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:42.541209 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-1d1a7b5a-3f55-4d66-ba1e-711f2ecec74b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c34969de69a4287a65ff0f4fe026b75', 'request_id': 'req-1d1a7b5a-3f55-4d66-ba1e-711f2ecec74b'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:42.664132 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-1d1a7b5a-3f55-4d66-ba1e-711f2ecec74b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c34969de69a4287a65ff0f4fe026b75', 'request_id': 'req-1d1a7b5a-3f55-4d66-ba1e-711f2ecec74b'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:42.675570 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-1d1a7b5a-3f55-4d66-ba1e-711f2ecec74b admin admin] The token cache is being invalidate because project 8c34969de69a4287a65ff0f4fe026b75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:14:42.675971 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-1d1a7b5a-3f55-4d66-ba1e-711f2ecec74b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:42.676655 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-1d1a7b5a-3f55-4d66-ba1e-711f2ecec74b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:42.678457 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1081/2211] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:14:42 2022] DELETE /identity/v3/projects/8c34969de69a4287a65ff0f4fe026b75 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:42.682326 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b7eca7-d353-46ca-b186-8dfff2edb35c admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:42.683903 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-89b7eca7-d353-46ca-b186-8dfff2edb35c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:42.697298 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b7eca7-d353-46ca-b186-8dfff2edb35c admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:42.756040 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d96e13-50dd-4ac3-b1d5-1bfa96c152a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:42.756932 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d96e13-50dd-4ac3-b1d5-1bfa96c152a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:42.757220 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d96e13-50dd-4ac3-b1d5-1bfa96c152a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:42.757498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d96e13-50dd-4ac3-b1d5-1bfa96c152a0 admin admin] PATH_INFO: `/v3/users/eddd17f7c2864e4bb38b440d9ec62e75` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:42.758701 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d96e13-50dd-4ac3-b1d5-1bfa96c152a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eddd17f7c2864e4bb38b440d9ec62e75)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:42.760711 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d96e13-50dd-4ac3-b1d5-1bfa96c152a0 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:42.983012 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-d4d96e13-50dd-4ac3-b1d5-1bfa96c152a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eddd17f7c2864e4bb38b440d9ec62e75'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:42.985948 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:42.993459 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-d4d96e13-50dd-4ac3-b1d5-1bfa96c152a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eddd17f7c2864e4bb38b440d9ec62e75'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:42.994283 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-d4d96e13-50dd-4ac3-b1d5-1bfa96c152a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eddd17f7c2864e4bb38b440d9ec62e75'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:43.012732 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-d4d96e13-50dd-4ac3-b1d5-1bfa96c152a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eddd17f7c2864e4bb38b440d9ec62e75'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:43.078773 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-d4d96e13-50dd-4ac3-b1d5-1bfa96c152a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eddd17f7c2864e4bb38b440d9ec62e75'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:43.086044 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1131/2212] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:14:42 2022] DELETE /identity/v3/users/eddd17f7c2864e4bb38b440d9ec62e75 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:43.695679 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1a7b5a-3f55-4d66-ba1e-711f2ecec74b admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:43.695679 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-1d1a7b5a-3f55-4d66-ba1e-711f2ecec74b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:43.730701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1a7b5a-3f55-4d66-ba1e-711f2ecec74b admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:43.814613 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e434d9d-9fac-4066-a5c6-fb11881fd3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:43.815483 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e434d9d-9fac-4066-a5c6-fb11881fd3da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:43.815834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e434d9d-9fac-4066-a5c6-fb11881fd3da admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:43.816125 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e434d9d-9fac-4066-a5c6-fb11881fd3da admin admin] PATH_INFO: `/v3/projects/6c029de4dda94aabba4ab5c0ddd45b90` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:43.817384 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e434d9d-9fac-4066-a5c6-fb11881fd3da admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c029de4dda94aabba4ab5c0ddd45b90)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:43.819458 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e434d9d-9fac-4066-a5c6-fb11881fd3da admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:43.985894 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-4e434d9d-9fac-4066-a5c6-fb11881fd3da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c029de4dda94aabba4ab5c0ddd45b90', 'request_id': 'req-4e434d9d-9fac-4066-a5c6-fb11881fd3da'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:43.992425 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-4e434d9d-9fac-4066-a5c6-fb11881fd3da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c029de4dda94aabba4ab5c0ddd45b90', 'request_id': 'req-4e434d9d-9fac-4066-a5c6-fb11881fd3da'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:44.074468 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-4e434d9d-9fac-4066-a5c6-fb11881fd3da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c029de4dda94aabba4ab5c0ddd45b90', 'request_id': 'req-4e434d9d-9fac-4066-a5c6-fb11881fd3da'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:44.088887 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-4e434d9d-9fac-4066-a5c6-fb11881fd3da admin admin] The token cache is being invalidate because project 6c029de4dda94aabba4ab5c0ddd45b90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:14:44.089181 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-4e434d9d-9fac-4066-a5c6-fb11881fd3da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:44.090127 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-4e434d9d-9fac-4066-a5c6-fb11881fd3da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:44.094573 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1082/2213] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:14:43 2022] DELETE /identity/v3/projects/6c029de4dda94aabba4ab5c0ddd45b90 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:44.100037 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42dce0a-0760-477d-9c46-471da12080c8 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:44.103545 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42dce0a-0760-477d-9c46-471da12080c8 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:44.103545 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42dce0a-0760-477d-9c46-471da12080c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:44.144456 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-e42dce0a-0760-477d-9c46-471da12080c8 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:44.146500 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-e42dce0a-0760-477d-9c46-471da12080c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:44.220650 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1132/2214] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:14:44.229464 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b918bdb-284b-4863-913c-70bd88580a3e None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:44.229761 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b918bdb-284b-4863-913c-70bd88580a3e None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:44.230048 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b918bdb-284b-4863-913c-70bd88580a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:44.250118 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-2b918bdb-284b-4863-913c-70bd88580a3e None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:44.251240 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-2b918bdb-284b-4863-913c-70bd88580a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:44.262693 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1083/2215] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:14:44.275895 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42dce0a-0760-477d-9c46-471da12080c8 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:44.288610 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42dce0a-0760-477d-9c46-471da12080c8 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:44.294706 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca3b64b-706d-4416-ae53-5ff20bff715b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:44.296179 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca3b64b-706d-4416-ae53-5ff20bff715b admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:44.296521 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca3b64b-706d-4416-ae53-5ff20bff715b admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:44.296813 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca3b64b-706d-4416-ae53-5ff20bff715b admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:44.297561 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca3b64b-706d-4416-ae53-5ff20bff715b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:14:44.297895 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca3b64b-706d-4416-ae53-5ff20bff715b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:44.299944 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca3b64b-706d-4416-ae53-5ff20bff715b admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:44.318656 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1133/2216] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:14:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:14:44.323650 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b918bdb-284b-4863-913c-70bd88580a3e None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:44.340798 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b918bdb-284b-4863-913c-70bd88580a3e None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:44.343133 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc50e6f-43b3-41fc-842d-83e09dc67bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:44.343907 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc50e6f-43b3-41fc-842d-83e09dc67bc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:44.344203 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc50e6f-43b3-41fc-842d-83e09dc67bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:44.344485 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc50e6f-43b3-41fc-842d-83e09dc67bc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:44.345276 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc50e6f-43b3-41fc-842d-83e09dc67bc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:44.347414 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc50e6f-43b3-41fc-842d-83e09dc67bc4 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:44.359351 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:44.419909 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca3b64b-706d-4416-ae53-5ff20bff715b admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:44.423253 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-1ca3b64b-706d-4416-ae53-5ff20bff715b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:44.439605 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca3b64b-706d-4416-ae53-5ff20bff715b admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:44.446092 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eff930c-0b00-46db-bb58-ca2774527406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:44.449625 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1084/2217] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:14:44 2022] POST /identity/v3/projects => generated 369 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:14:44.449985 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff930c-0b00-46db-bb58-ca2774527406 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:44.449985 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff930c-0b00-46db-bb58-ca2774527406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:44.449985 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eff930c-0b00-46db-bb58-ca2774527406 admin admin] PATH_INFO: `/v3/users/8e72875df16d497f9a542da0a5636f5f` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:44.450297 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eff930c-0b00-46db-bb58-ca2774527406 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e72875df16d497f9a542da0a5636f5f)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:44.450989 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc50e6f-43b3-41fc-842d-83e09dc67bc4 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:44.453104 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eff930c-0b00-46db-bb58-ca2774527406 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:44.526059 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc50e6f-43b3-41fc-842d-83e09dc67bc4 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:44.531807 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba2640-38eb-47a3-a8fe-315e3417773c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:44.532494 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba2640-38eb-47a3-a8fe-315e3417773c admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:44.532782 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba2640-38eb-47a3-a8fe-315e3417773c admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:44.538135 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fba2640-38eb-47a3-a8fe-315e3417773c admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:44.538135 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fba2640-38eb-47a3-a8fe-315e3417773c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:44.538135 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fba2640-38eb-47a3-a8fe-315e3417773c admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:44.659628 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1085/2218] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:14:44 2022] POST /identity/v3/users => generated 285 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:14:44.663914 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba2640-38eb-47a3-a8fe-315e3417773c admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:44.673182 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fba2640-38eb-47a3-a8fe-315e3417773c admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:44.679923 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90e16e-0b97-4c8d-9271-ed1e6005cc8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:44.680590 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90e16e-0b97-4c8d-9271-ed1e6005cc8f admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:44.681820 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90e16e-0b97-4c8d-9271-ed1e6005cc8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:44.681820 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90e16e-0b97-4c8d-9271-ed1e6005cc8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:44.682096 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc90e16e-0b97-4c8d-9271-ed1e6005cc8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:44.683516 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc90e16e-0b97-4c8d-9271-ed1e6005cc8f admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:44.691119 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0eff930c-0b00-46db-bb58-ca2774527406 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e72875df16d497f9a542da0a5636f5f'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:44.694280 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1086/2219] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:14:44.698453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0eff930c-0b00-46db-bb58-ca2774527406 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e72875df16d497f9a542da0a5636f5f'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:44.698616 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90e16e-0b97-4c8d-9271-ed1e6005cc8f admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:44.699298 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0eff930c-0b00-46db-bb58-ca2774527406 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e72875df16d497f9a542da0a5636f5f'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:44.709124 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90e16e-0b97-4c8d-9271-ed1e6005cc8f admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:44.712913 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0eff930c-0b00-46db-bb58-ca2774527406 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e72875df16d497f9a542da0a5636f5f'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:44.713999 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edda5a9-2c42-41bb-acb7-92d396cb52c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:44.714644 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edda5a9-2c42-41bb-acb7-92d396cb52c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:44.714932 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edda5a9-2c42-41bb-acb7-92d396cb52c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:44.715579 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edda5a9-2c42-41bb-acb7-92d396cb52c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:44.716395 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edda5a9-2c42-41bb-acb7-92d396cb52c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:44.717988 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edda5a9-2c42-41bb-acb7-92d396cb52c9 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:44.728971 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1087/2220] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:14:44.732970 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edda5a9-2c42-41bb-acb7-92d396cb52c9 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:44.742461 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edda5a9-2c42-41bb-acb7-92d396cb52c9 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:44.747074 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6b68d-e3a9-4977-9318-0830e525547c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:44.747963 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6b68d-e3a9-4977-9318-0830e525547c admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:44.748250 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6b68d-e3a9-4977-9318-0830e525547c admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:44.749060 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6b68d-e3a9-4977-9318-0830e525547c admin admin] PATH_INFO: `/v3/projects/9c32202a246c4bada91bcd9459df925d/users/2851b96fac754276b541b529cfbc59df/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:44.764521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e6b68d-e3a9-4977-9318-0830e525547c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c32202a246c4bada91bcd9459df925d, user_id=2851b96fac754276b541b529cfbc59df, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:44.767170 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e6b68d-e3a9-4977-9318-0830e525547c admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:44.789890 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-0eff930c-0b00-46db-bb58-ca2774527406 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e72875df16d497f9a542da0a5636f5f'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:44.796510 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1134/2221] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:14:44 2022] DELETE /identity/v3/users/8e72875df16d497f9a542da0a5636f5f => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:44.854656 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1088/2222] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:14:44 2022] PUT /identity/v3/projects/9c32202a246c4bada91bcd9459df925d/users/2851b96fac754276b541b529cfbc59df/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:14:44.865250 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0d6e5-5b52-4829-90b2-2a626f0521a1 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:44.865558 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0d6e5-5b52-4829-90b2-2a626f0521a1 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:44.865842 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0d6e5-5b52-4829-90b2-2a626f0521a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:44.908507 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-17b0d6e5-5b52-4829-90b2-2a626f0521a1 None None] MFA Rules not processed for user `2851b96fac754276b541b529cfbc59df`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:44.910116 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-17b0d6e5-5b52-4829-90b2-2a626f0521a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:44.928790 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:44.996602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1135/2223] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:44 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:45.009524 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6b68d-e3a9-4977-9318-0830e525547c admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:45.011327 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c4e6b68d-e3a9-4977-9318-0830e525547c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:45.080777 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e6b68d-e3a9-4977-9318-0830e525547c admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:45.084853 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6767db71-334e-42c4-bf16-6077b9936ab9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '635a410f471c43109438d0c39874070b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:45.085447 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6767db71-334e-42c4-bf16-6077b9936ab9 service glance] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:45.085768 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6767db71-334e-42c4-bf16-6077b9936ab9 service glance] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:45.086011 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6767db71-334e-42c4-bf16-6077b9936ab9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:45.094681 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6767db71-334e-42c4-bf16-6077b9936ab9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:45.096158 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6767db71-334e-42c4-bf16-6077b9936ab9 service glance] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:45.105017 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1089/2224] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:14:45 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:14:45.156632 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0d6e5-5b52-4829-90b2-2a626f0521a1 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:45.158100 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-17b0d6e5-5b52-4829-90b2-2a626f0521a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:45.239675 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0d6e5-5b52-4829-90b2-2a626f0521a1 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:45.244531 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a6dda1-4be9-4c4f-a7ba-6ac2c15d7997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:45.245493 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6dda1-4be9-4c4f-a7ba-6ac2c15d7997 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:45.245820 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6dda1-4be9-4c4f-a7ba-6ac2c15d7997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:45.246114 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6dda1-4be9-4c4f-a7ba-6ac2c15d7997 admin admin] PATH_INFO: `/v3/projects/ae7e64bccf304d5ebed6c23c90d47d7a` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:45.247451 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a6dda1-4be9-4c4f-a7ba-6ac2c15d7997 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae7e64bccf304d5ebed6c23c90d47d7a)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:45.250069 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a6dda1-4be9-4c4f-a7ba-6ac2c15d7997 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:45.400873 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-64a6dda1-4be9-4c4f-a7ba-6ac2c15d7997 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae7e64bccf304d5ebed6c23c90d47d7a', 'request_id': 'req-64a6dda1-4be9-4c4f-a7ba-6ac2c15d7997'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:45.408073 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-64a6dda1-4be9-4c4f-a7ba-6ac2c15d7997 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae7e64bccf304d5ebed6c23c90d47d7a', 'request_id': 'req-64a6dda1-4be9-4c4f-a7ba-6ac2c15d7997'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:45.413547 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-64a6dda1-4be9-4c4f-a7ba-6ac2c15d7997 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae7e64bccf304d5ebed6c23c90d47d7a', 'request_id': 'req-64a6dda1-4be9-4c4f-a7ba-6ac2c15d7997'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:45.480871 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-64a6dda1-4be9-4c4f-a7ba-6ac2c15d7997 admin admin] The token cache is being invalidate because project ae7e64bccf304d5ebed6c23c90d47d7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:14:45.481192 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-64a6dda1-4be9-4c4f-a7ba-6ac2c15d7997 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:45.481856 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-64a6dda1-4be9-4c4f-a7ba-6ac2c15d7997 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:45.484107 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1136/2225] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:14:45 2022] DELETE /identity/v3/projects/ae7e64bccf304d5ebed6c23c90d47d7a => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:45.489303 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0687c065-5e84-4125-8095-eca7e3ebfd10 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:45.489609 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0687c065-5e84-4125-8095-eca7e3ebfd10 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:45.489895 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0687c065-5e84-4125-8095-eca7e3ebfd10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:45.508003 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-0687c065-5e84-4125-8095-eca7e3ebfd10 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:45.509188 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-0687c065-5e84-4125-8095-eca7e3ebfd10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:45.542140 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:45.569139 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1090/2226] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:45.577815 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff76fda-6740-4503-833b-5b2eeaea0385 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:45.578106 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff76fda-6740-4503-833b-5b2eeaea0385 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:45.578392 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff76fda-6740-4503-833b-5b2eeaea0385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:45.596925 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-0ff76fda-6740-4503-833b-5b2eeaea0385 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:45.598043 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-0ff76fda-6740-4503-833b-5b2eeaea0385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:45.606084 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1137/2227] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:45.610720 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0687c065-5e84-4125-8095-eca7e3ebfd10 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:45.620176 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0687c065-5e84-4125-8095-eca7e3ebfd10 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:45.624708 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc512304-b9ee-40a3-9455-595873fc09c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:45.625354 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc512304-b9ee-40a3-9455-595873fc09c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:45.625635 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc512304-b9ee-40a3-9455-595873fc09c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:45.625910 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc512304-b9ee-40a3-9455-595873fc09c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:45.626629 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc512304-b9ee-40a3-9455-595873fc09c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:14:45.626935 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc512304-b9ee-40a3-9455-595873fc09c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:45.628641 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc512304-b9ee-40a3-9455-595873fc09c4 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:45.640272 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1091/2228] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:14:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:14:45.644449 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff76fda-6740-4503-833b-5b2eeaea0385 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:45.654234 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff76fda-6740-4503-833b-5b2eeaea0385 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:45.661098 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efac2a53-587b-4e23-9ef4-62e69e58c630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:45.661098 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac2a53-587b-4e23-9ef4-62e69e58c630 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:45.661098 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac2a53-587b-4e23-9ef4-62e69e58c630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:45.661098 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac2a53-587b-4e23-9ef4-62e69e58c630 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:45.661727 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efac2a53-587b-4e23-9ef4-62e69e58c630 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:45.663651 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efac2a53-587b-4e23-9ef4-62e69e58c630 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:45.769444 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1138/2229] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:14:45 2022] POST /identity/v3/projects => generated 387 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:14:45.779572 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc512304-b9ee-40a3-9455-595873fc09c4 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:45.790106 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc512304-b9ee-40a3-9455-595873fc09c4 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:45.846733 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aff9e7-d203-4a46-84c8-ee41cc7eac6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:45.847463 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aff9e7-d203-4a46-84c8-ee41cc7eac6e admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:45.847704 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aff9e7-d203-4a46-84c8-ee41cc7eac6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:45.848003 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aff9e7-d203-4a46-84c8-ee41cc7eac6e admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:45.848690 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aff9e7-d203-4a46-84c8-ee41cc7eac6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:45.850603 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aff9e7-d203-4a46-84c8-ee41cc7eac6e admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:45.968810 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1092/2230] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:14:45 2022] POST /identity/v3/users => generated 294 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:14:45.981633 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efac2a53-587b-4e23-9ef4-62e69e58c630 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:45.992043 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efac2a53-587b-4e23-9ef4-62e69e58c630 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:45.996628 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d1c95a-f3a6-471d-8956-42730628f478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:46.002071 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d1c95a-f3a6-471d-8956-42730628f478 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:46.002071 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d1c95a-f3a6-471d-8956-42730628f478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:46.002071 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d1c95a-f3a6-471d-8956-42730628f478 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:46.002071 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d1c95a-f3a6-471d-8956-42730628f478 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:46.002071 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d1c95a-f3a6-471d-8956-42730628f478 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:46.010612 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1139/2231] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:14:46.015074 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aff9e7-d203-4a46-84c8-ee41cc7eac6e admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:46.025254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aff9e7-d203-4a46-84c8-ee41cc7eac6e admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:46.030894 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00e6a76-1d56-4773-8a5e-7a62066e435f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:46.031562 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e6a76-1d56-4773-8a5e-7a62066e435f admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:46.031889 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e6a76-1d56-4773-8a5e-7a62066e435f admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:46.032175 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00e6a76-1d56-4773-8a5e-7a62066e435f admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:46.032902 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00e6a76-1d56-4773-8a5e-7a62066e435f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:46.034505 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00e6a76-1d56-4773-8a5e-7a62066e435f admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:46.044473 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1093/2232] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:14:46.048638 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d1c95a-f3a6-471d-8956-42730628f478 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:46.058510 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d1c95a-f3a6-471d-8956-42730628f478 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:46.063072 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb266983-26f2-49a8-a589-b49387a7940d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:46.063975 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb266983-26f2-49a8-a589-b49387a7940d admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:46.064262 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb266983-26f2-49a8-a589-b49387a7940d admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:46.064545 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb266983-26f2-49a8-a589-b49387a7940d admin admin] PATH_INFO: `/v3/projects/b1d0f8e711df49f58773fd76227ee878/users/8b5fdc57573c421195439a0c8e95d7d4/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:46.066748 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:46.079896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb266983-26f2-49a8-a589-b49387a7940d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1d0f8e711df49f58773fd76227ee878, user_id=8b5fdc57573c421195439a0c8e95d7d4, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:46.082518 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb266983-26f2-49a8-a589-b49387a7940d admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:46.199569 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1140/2233] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:14:46 2022] PUT /identity/v3/projects/b1d0f8e711df49f58773fd76227ee878/users/8b5fdc57573c421195439a0c8e95d7d4/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:14:46.208253 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10823b89-d50b-4ddc-87e3-be228fdc6bbe None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:46.208549 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10823b89-d50b-4ddc-87e3-be228fdc6bbe None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:46.208889 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10823b89-d50b-4ddc-87e3-be228fdc6bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:46.228088 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-10823b89-d50b-4ddc-87e3-be228fdc6bbe None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:46.228088 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-10823b89-d50b-4ddc-87e3-be228fdc6bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:46.291031 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1094/2234] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:14:46.300829 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb266983-26f2-49a8-a589-b49387a7940d admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:46.302577 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-cb266983-26f2-49a8-a589-b49387a7940d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:46.367692 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb266983-26f2-49a8-a589-b49387a7940d admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:46.371351 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efb6a94-12c3-4e4f-81da-003be275a483 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:46.372056 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb6a94-12c3-4e4f-81da-003be275a483 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:46.372341 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb6a94-12c3-4e4f-81da-003be275a483 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:46.377292 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efb6a94-12c3-4e4f-81da-003be275a483 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:46.381769 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efb6a94-12c3-4e4f-81da-003be275a483 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:46.383288 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efb6a94-12c3-4e4f-81da-003be275a483 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:46.392716 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1141/2235] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:14:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:14:47.386023 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10823b89-d50b-4ddc-87e3-be228fdc6bbe None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:47.388984 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-10823b89-d50b-4ddc-87e3-be228fdc6bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:47.417692 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10823b89-d50b-4ddc-87e3-be228fdc6bbe None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:47.429878 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56717b71-6510-43bc-b5c6-e65626a55146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:47.431152 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56717b71-6510-43bc-b5c6-e65626a55146 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:47.431541 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56717b71-6510-43bc-b5c6-e65626a55146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:47.434132 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56717b71-6510-43bc-b5c6-e65626a55146 admin admin] PATH_INFO: `/v3/users/2d6e29ab397e4f51a4695069895395dc` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:47.437693 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56717b71-6510-43bc-b5c6-e65626a55146 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d6e29ab397e4f51a4695069895395dc)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:47.439792 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56717b71-6510-43bc-b5c6-e65626a55146 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:47.677673 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:47.683491 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-56717b71-6510-43bc-b5c6-e65626a55146 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d6e29ab397e4f51a4695069895395dc'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:47.695624 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-56717b71-6510-43bc-b5c6-e65626a55146 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d6e29ab397e4f51a4695069895395dc'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:47.708085 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-56717b71-6510-43bc-b5c6-e65626a55146 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d6e29ab397e4f51a4695069895395dc'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:47.709457 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-56717b71-6510-43bc-b5c6-e65626a55146 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d6e29ab397e4f51a4695069895395dc'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:47.807954 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-56717b71-6510-43bc-b5c6-e65626a55146 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d6e29ab397e4f51a4695069895395dc'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:47.817287 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1095/2236] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:14:47 2022] DELETE /identity/v3/users/2d6e29ab397e4f51a4695069895395dc => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:47.821541 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8abf4-c004-4928-9903-dee63547bfb0 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:47.822379 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8abf4-c004-4928-9903-dee63547bfb0 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:47.824080 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8abf4-c004-4928-9903-dee63547bfb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:47.841304 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-aeb8abf4-c004-4928-9903-dee63547bfb0 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:47.842532 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-aeb8abf4-c004-4928-9903-dee63547bfb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:47.907819 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1142/2237] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:47.917862 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56717b71-6510-43bc-b5c6-e65626a55146 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:47.919326 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-56717b71-6510-43bc-b5c6-e65626a55146 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:47.993532 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56717b71-6510-43bc-b5c6-e65626a55146 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:47.997265 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeb1090-66d1-4f8f-9189-fd5bd1fba539 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:47.997974 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb1090-66d1-4f8f-9189-fd5bd1fba539 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:47.998204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb1090-66d1-4f8f-9189-fd5bd1fba539 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:47.998485 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feeb1090-66d1-4f8f-9189-fd5bd1fba539 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:48.009305 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeb1090-66d1-4f8f-9189-fd5bd1fba539 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:48.010747 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feeb1090-66d1-4f8f-9189-fd5bd1fba539 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:48.025700 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1096/2238] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:14:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:14:49.058132 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb8abf4-c004-4928-9903-dee63547bfb0 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:49.059681 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-aeb8abf4-c004-4928-9903-dee63547bfb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:49.073639 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb8abf4-c004-4928-9903-dee63547bfb0 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:49.081169 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba7e295-5bf3-4e98-ad2b-8bfa6bc02dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:49.085150 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba7e295-5bf3-4e98-ad2b-8bfa6bc02dfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:49.085150 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba7e295-5bf3-4e98-ad2b-8bfa6bc02dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:49.085150 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba7e295-5bf3-4e98-ad2b-8bfa6bc02dfe admin admin] PATH_INFO: `/v3/projects/e17a8cb8565d4a80be3a244fa6d71dad` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:49.085150 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba7e295-5bf3-4e98-ad2b-8bfa6bc02dfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=e17a8cb8565d4a80be3a244fa6d71dad)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:49.086428 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba7e295-5bf3-4e98-ad2b-8bfa6bc02dfe admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:49.204438 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-4ba7e295-5bf3-4e98-ad2b-8bfa6bc02dfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e17a8cb8565d4a80be3a244fa6d71dad', 'request_id': 'req-4ba7e295-5bf3-4e98-ad2b-8bfa6bc02dfe'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:49.210568 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-4ba7e295-5bf3-4e98-ad2b-8bfa6bc02dfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e17a8cb8565d4a80be3a244fa6d71dad', 'request_id': 'req-4ba7e295-5bf3-4e98-ad2b-8bfa6bc02dfe'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:49.216261 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-4ba7e295-5bf3-4e98-ad2b-8bfa6bc02dfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e17a8cb8565d4a80be3a244fa6d71dad', 'request_id': 'req-4ba7e295-5bf3-4e98-ad2b-8bfa6bc02dfe'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:49.345670 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-4ba7e295-5bf3-4e98-ad2b-8bfa6bc02dfe admin admin] The token cache is being invalidate because project e17a8cb8565d4a80be3a244fa6d71dad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:14:49.346058 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-4ba7e295-5bf3-4e98-ad2b-8bfa6bc02dfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:49.346827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-4ba7e295-5bf3-4e98-ad2b-8bfa6bc02dfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:49.348704 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1143/2239] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:14:49 2022] DELETE /identity/v3/projects/e17a8cb8565d4a80be3a244fa6d71dad => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:49.352722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeb1090-66d1-4f8f-9189-fd5bd1fba539 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:49.363437 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feeb1090-66d1-4f8f-9189-fd5bd1fba539 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:49.447601 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e8b96c-53d6-4103-be24-c1e939db56d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:49.448474 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8b96c-53d6-4103-be24-c1e939db56d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:49.448767 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8b96c-53d6-4103-be24-c1e939db56d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:49.449050 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8b96c-53d6-4103-be24-c1e939db56d2 admin admin] PATH_INFO: `/v3/users/d52ffade10eb49dba76006657964164d` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:49.450459 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e8b96c-53d6-4103-be24-c1e939db56d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d52ffade10eb49dba76006657964164d)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:49.453005 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e8b96c-53d6-4103-be24-c1e939db56d2 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:49.691899 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-08e8b96c-53d6-4103-be24-c1e939db56d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd52ffade10eb49dba76006657964164d'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:49.714045 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:49.727664 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-08e8b96c-53d6-4103-be24-c1e939db56d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd52ffade10eb49dba76006657964164d'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:49.745103 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-08e8b96c-53d6-4103-be24-c1e939db56d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd52ffade10eb49dba76006657964164d'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:49.746882 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-08e8b96c-53d6-4103-be24-c1e939db56d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd52ffade10eb49dba76006657964164d'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:49.831354 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-08e8b96c-53d6-4103-be24-c1e939db56d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd52ffade10eb49dba76006657964164d'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:49.843618 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1097/2240] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:14:49 2022] DELETE /identity/v3/users/d52ffade10eb49dba76006657964164d => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:50.541846 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba7e295-5bf3-4e98-ad2b-8bfa6bc02dfe admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:50.544173 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-4ba7e295-5bf3-4e98-ad2b-8bfa6bc02dfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:50.589916 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba7e295-5bf3-4e98-ad2b-8bfa6bc02dfe admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:50.598548 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:50.679483 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8cbbb4-bf7e-4707-ac79-54ce35b21ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:50.680410 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8cbbb4-bf7e-4707-ac79-54ce35b21ecd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:50.686765 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8cbbb4-bf7e-4707-ac79-54ce35b21ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:50.686765 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8cbbb4-bf7e-4707-ac79-54ce35b21ecd admin admin] PATH_INFO: `/v3/projects/6d366bf0aae84c95adfff19b95045648` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:50.686765 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8cbbb4-bf7e-4707-ac79-54ce35b21ecd admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d366bf0aae84c95adfff19b95045648)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:50.686765 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8cbbb4-bf7e-4707-ac79-54ce35b21ecd admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:50.808362 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1d8cbbb4-bf7e-4707-ac79-54ce35b21ecd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d366bf0aae84c95adfff19b95045648', 'request_id': 'req-1d8cbbb4-bf7e-4707-ac79-54ce35b21ecd'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:50.816304 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1d8cbbb4-bf7e-4707-ac79-54ce35b21ecd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d366bf0aae84c95adfff19b95045648', 'request_id': 'req-1d8cbbb4-bf7e-4707-ac79-54ce35b21ecd'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:50.822551 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1d8cbbb4-bf7e-4707-ac79-54ce35b21ecd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d366bf0aae84c95adfff19b95045648', 'request_id': 'req-1d8cbbb4-bf7e-4707-ac79-54ce35b21ecd'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:50.877092 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1d8cbbb4-bf7e-4707-ac79-54ce35b21ecd admin admin] The token cache is being invalidate because project 6d366bf0aae84c95adfff19b95045648 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:14:50.877454 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1d8cbbb4-bf7e-4707-ac79-54ce35b21ecd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:50.879877 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1d8cbbb4-bf7e-4707-ac79-54ce35b21ecd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:50.881486 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1144/2241] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:14:50 2022] DELETE /identity/v3/projects/6d366bf0aae84c95adfff19b95045648 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:50.886454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c47b3-2938-4383-a5fe-9f4bfe4e0ff7 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:50.886932 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c47b3-2938-4383-a5fe-9f4bfe4e0ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:50.887034 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c47b3-2938-4383-a5fe-9f4bfe4e0ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:50.908014 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-ae0c47b3-2938-4383-a5fe-9f4bfe4e0ff7 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:50.909638 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ae0c47b3-2938-4383-a5fe-9f4bfe4e0ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:51.036576 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1098/2242] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:51.038115 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2127149-f0b7-49c6-a9bd-534b584b0933 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:51.038410 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2127149-f0b7-49c6-a9bd-534b584b0933 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:51.038685 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2127149-f0b7-49c6-a9bd-534b584b0933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:51.069225 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-f2127149-f0b7-49c6-a9bd-534b584b0933 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:51.070577 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f2127149-f0b7-49c6-a9bd-534b584b0933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:51.085862 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1145/2243] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:14:51.087631 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c47b3-2938-4383-a5fe-9f4bfe4e0ff7 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:51.138819 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c47b3-2938-4383-a5fe-9f4bfe4e0ff7 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:51.155351 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c71aac-a62c-408d-a4dd-b59e8492df79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:51.156865 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c71aac-a62c-408d-a4dd-b59e8492df79 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:51.157369 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c71aac-a62c-408d-a4dd-b59e8492df79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:51.157835 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c71aac-a62c-408d-a4dd-b59e8492df79 admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:51.158996 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c71aac-a62c-408d-a4dd-b59e8492df79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:14:51.159518 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c71aac-a62c-408d-a4dd-b59e8492df79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:51.161668 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c71aac-a62c-408d-a4dd-b59e8492df79 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:51.180882 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1099/2244] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:14:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:14:51.184442 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2127149-f0b7-49c6-a9bd-534b584b0933 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:51.205346 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2127149-f0b7-49c6-a9bd-534b584b0933 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:51.213861 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d9b419-aeeb-467b-b378-87650fa7786d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:51.215321 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9b419-aeeb-467b-b378-87650fa7786d admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:51.215956 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9b419-aeeb-467b-b378-87650fa7786d admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:51.217132 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d9b419-aeeb-467b-b378-87650fa7786d admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:51.219073 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d9b419-aeeb-467b-b378-87650fa7786d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:51.222340 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d9b419-aeeb-467b-b378-87650fa7786d admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:51.350898 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1146/2245] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:14:51 2022] POST /identity/v3/projects => generated 401 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:14:51.355074 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c71aac-a62c-408d-a4dd-b59e8492df79 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:51.366567 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c71aac-a62c-408d-a4dd-b59e8492df79 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:51.458826 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ac87a-b688-48c4-848e-8f72c5889983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:51.459735 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ac87a-b688-48c4-848e-8f72c5889983 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:51.460031 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ac87a-b688-48c4-848e-8f72c5889983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:51.460305 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406ac87a-b688-48c4-848e-8f72c5889983 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:51.460998 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ac87a-b688-48c4-848e-8f72c5889983 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:51.462924 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406ac87a-b688-48c4-848e-8f72c5889983 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:51.574338 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1100/2246] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:14:51 2022] POST /identity/v3/users => generated 301 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:14:51.578562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d9b419-aeeb-467b-b378-87650fa7786d admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:51.591110 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d9b419-aeeb-467b-b378-87650fa7786d admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:51.597005 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379b8614-3283-4cc0-8790-fde3f5d5b70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:51.598400 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b8614-3283-4cc0-8790-fde3f5d5b70c admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:51.598800 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b8614-3283-4cc0-8790-fde3f5d5b70c admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:51.599303 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379b8614-3283-4cc0-8790-fde3f5d5b70c admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:51.600320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379b8614-3283-4cc0-8790-fde3f5d5b70c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:51.605776 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379b8614-3283-4cc0-8790-fde3f5d5b70c admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:51.615794 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1147/2247] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:14:51.622088 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ac87a-b688-48c4-848e-8f72c5889983 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:51.633659 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406ac87a-b688-48c4-848e-8f72c5889983 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:51.639120 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1b2cc7-bbce-49dc-9423-f645c5fc6456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:51.639997 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b2cc7-bbce-49dc-9423-f645c5fc6456 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:51.640387 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b2cc7-bbce-49dc-9423-f645c5fc6456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:51.641128 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1b2cc7-bbce-49dc-9423-f645c5fc6456 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:51.641627 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1b2cc7-bbce-49dc-9423-f645c5fc6456 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:51.643357 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1b2cc7-bbce-49dc-9423-f645c5fc6456 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:51.655870 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1101/2248] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:14:51.667262 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379b8614-3283-4cc0-8790-fde3f5d5b70c admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:51.679572 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379b8614-3283-4cc0-8790-fde3f5d5b70c admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:51.686863 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28f0702-19c8-456b-89cd-65089e2dd7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:51.687983 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f0702-19c8-456b-89cd-65089e2dd7b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:51.688277 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f0702-19c8-456b-89cd-65089e2dd7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:51.692053 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28f0702-19c8-456b-89cd-65089e2dd7b3 admin admin] PATH_INFO: `/v3/projects/73dca3c38dc24c01be6a39587d9a8e44/users/d221de5786e64a49b858ce7a593fee4c/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:51.707180 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28f0702-19c8-456b-89cd-65089e2dd7b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73dca3c38dc24c01be6a39587d9a8e44, user_id=d221de5786e64a49b858ce7a593fee4c, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:51.723093 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28f0702-19c8-456b-89cd-65089e2dd7b3 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:51.861229 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1148/2249] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:14:51 2022] PUT /identity/v3/projects/73dca3c38dc24c01be6a39587d9a8e44/users/d221de5786e64a49b858ce7a593fee4c/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:51.864144 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09454fcd-5b3f-4ed4-8d79-725ed07b2958 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:51.864454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09454fcd-5b3f-4ed4-8d79-725ed07b2958 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:51.864881 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09454fcd-5b3f-4ed4-8d79-725ed07b2958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:51.911062 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-09454fcd-5b3f-4ed4-8d79-725ed07b2958 None None] MFA Rules not processed for user `d221de5786e64a49b858ce7a593fee4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:51.912413 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-09454fcd-5b3f-4ed4-8d79-725ed07b2958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:51.949949 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:52.007203 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1102/2250] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:51 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:52.011319 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28f0702-19c8-456b-89cd-65089e2dd7b3 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:52.069570 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:52.097219 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28f0702-19c8-456b-89cd-65089e2dd7b3 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:52.102266 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25109413-c4ef-4a0a-af36-7e614ffc7dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:52.103034 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25109413-c4ef-4a0a-af36-7e614ffc7dbc admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:52.103327 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25109413-c4ef-4a0a-af36-7e614ffc7dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:52.103612 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25109413-c4ef-4a0a-af36-7e614ffc7dbc admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:52.104664 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25109413-c4ef-4a0a-af36-7e614ffc7dbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:52.106409 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25109413-c4ef-4a0a-af36-7e614ffc7dbc admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:52.220695 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1149/2251] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:14:52 2022] POST /identity/v3/projects => generated 399 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:14:52.225553 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09454fcd-5b3f-4ed4-8d79-725ed07b2958 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:52.252423 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09454fcd-5b3f-4ed4-8d79-725ed07b2958 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:52.321778 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154afe9e-7141-4194-b14b-efb13d9c7402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:52.322553 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154afe9e-7141-4194-b14b-efb13d9c7402 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:52.322912 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154afe9e-7141-4194-b14b-efb13d9c7402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:52.323444 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154afe9e-7141-4194-b14b-efb13d9c7402 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:52.324310 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154afe9e-7141-4194-b14b-efb13d9c7402 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:52.326688 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154afe9e-7141-4194-b14b-efb13d9c7402 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:52.450872 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1103/2252] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:14:52 2022] POST /identity/v3/users => generated 306 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:14:52.454852 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25109413-c4ef-4a0a-af36-7e614ffc7dbc admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:52.466972 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25109413-c4ef-4a0a-af36-7e614ffc7dbc admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:52.472165 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3536f96-d42b-4d08-ae46-b3d7581962b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:52.472852 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3536f96-d42b-4d08-ae46-b3d7581962b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:52.473140 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3536f96-d42b-4d08-ae46-b3d7581962b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:52.473425 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3536f96-d42b-4d08-ae46-b3d7581962b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:52.474451 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3536f96-d42b-4d08-ae46-b3d7581962b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:52.476128 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3536f96-d42b-4d08-ae46-b3d7581962b9 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:52.488307 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1150/2253] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:14:52.492890 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154afe9e-7141-4194-b14b-efb13d9c7402 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:52.506580 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154afe9e-7141-4194-b14b-efb13d9c7402 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:52.510770 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed439cd8-c04e-42a2-ba49-ef5a4971797d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:52.511691 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed439cd8-c04e-42a2-ba49-ef5a4971797d admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:52.512885 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed439cd8-c04e-42a2-ba49-ef5a4971797d admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:52.513172 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed439cd8-c04e-42a2-ba49-ef5a4971797d admin admin] PATH_INFO: `/v3/projects/d1dad978a7674f3a978c55e7ad4079dc/users/41d9e34d7689447d97ca7f33037ba711/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:52.573992 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed439cd8-c04e-42a2-ba49-ef5a4971797d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1dad978a7674f3a978c55e7ad4079dc, user_id=41d9e34d7689447d97ca7f33037ba711, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:52.586658 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed439cd8-c04e-42a2-ba49-ef5a4971797d admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:52.673937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1104/2254] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:14:52 2022] PUT /identity/v3/projects/d1dad978a7674f3a978c55e7ad4079dc/users/41d9e34d7689447d97ca7f33037ba711/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:52.675202 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3536f96-d42b-4d08-ae46-b3d7581962b9 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:52.685674 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3536f96-d42b-4d08-ae46-b3d7581962b9 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:52.818009 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454cbd89-cc93-46c0-8da4-1a64a80cb38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:52.818801 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454cbd89-cc93-46c0-8da4-1a64a80cb38e admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:52.819186 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454cbd89-cc93-46c0-8da4-1a64a80cb38e admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:52.819572 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454cbd89-cc93-46c0-8da4-1a64a80cb38e admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:52.820624 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454cbd89-cc93-46c0-8da4-1a64a80cb38e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:52.822353 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454cbd89-cc93-46c0-8da4-1a64a80cb38e admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:52.844749 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1151/2255] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:14:52 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:14:52.846363 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed439cd8-c04e-42a2-ba49-ef5a4971797d admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:52.857543 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed439cd8-c04e-42a2-ba49-ef5a4971797d admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:52.863057 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7236dc1-7bf0-49c5-ba8b-167ed2d9686c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:52.863979 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7236dc1-7bf0-49c5-ba8b-167ed2d9686c admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:52.864261 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7236dc1-7bf0-49c5-ba8b-167ed2d9686c admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:52.864532 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7236dc1-7bf0-49c5-ba8b-167ed2d9686c admin admin] PATH_INFO: `/v3/projects/d1dad978a7674f3a978c55e7ad4079dc/users/41d9e34d7689447d97ca7f33037ba711/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:52.866707 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7236dc1-7bf0-49c5-ba8b-167ed2d9686c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1dad978a7674f3a978c55e7ad4079dc, user_id=41d9e34d7689447d97ca7f33037ba711, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:52.869732 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7236dc1-7bf0-49c5-ba8b-167ed2d9686c admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:52.881568 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1105/2256] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:14:52 2022] PUT /identity/v3/projects/d1dad978a7674f3a978c55e7ad4079dc/users/41d9e34d7689447d97ca7f33037ba711/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:52.892759 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64114d07-ddae-4910-8638-22880b243a53 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:52.893106 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64114d07-ddae-4910-8638-22880b243a53 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:52.893339 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64114d07-ddae-4910-8638-22880b243a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:52.927910 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-64114d07-ddae-4910-8638-22880b243a53 None None] MFA Rules not processed for user `41d9e34d7689447d97ca7f33037ba711`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:52.929515 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-64114d07-ddae-4910-8638-22880b243a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:52.959324 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1f89c-2ad4-4171-a6cd-fb5bf6bf2407 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:52.959923 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1f89c-2ad4-4171-a6cd-fb5bf6bf2407 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:52.959923 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba1f89c-2ad4-4171-a6cd-fb5bf6bf2407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:52.993323 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-4ba1f89c-2ad4-4171-a6cd-fb5bf6bf2407 None None] MFA Rules not processed for user `8b5fdc57573c421195439a0c8e95d7d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:52.996153 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4ba1f89c-2ad4-4171-a6cd-fb5bf6bf2407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:53.028710 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1152/2257] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:52 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:53.043883 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64114d07-ddae-4910-8638-22880b243a53 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:53.045535 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-64114d07-ddae-4910-8638-22880b243a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:53.084865 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1106/2258] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:14:53.099696 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba1f89c-2ad4-4171-a6cd-fb5bf6bf2407 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:53.101154 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4ba1f89c-2ad4-4171-a6cd-fb5bf6bf2407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:53.114010 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64114d07-ddae-4910-8638-22880b243a53 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:53.116425 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:53.125910 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f9616b-3915-4e9c-a9bf-596aa19aa3de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:53.125910 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f9616b-3915-4e9c-a9bf-596aa19aa3de service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:53.125910 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f9616b-3915-4e9c-a9bf-596aa19aa3de service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:53.125910 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f9616b-3915-4e9c-a9bf-596aa19aa3de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:53.126305 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:53.132039 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f9616b-3915-4e9c-a9bf-596aa19aa3de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:53.133529 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f9616b-3915-4e9c-a9bf-596aa19aa3de service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:53.144537 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1153/2259] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:14:53 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 17:14:53.192927 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba1f89c-2ad4-4171-a6cd-fb5bf6bf2407 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:53.199793 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed37bd0-3dbd-4b2a-b33b-b52b255b89cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:53.206762 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed37bd0-3dbd-4b2a-b33b-b52b255b89cd service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:53.206762 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed37bd0-3dbd-4b2a-b33b-b52b255b89cd service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:53.206762 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed37bd0-3dbd-4b2a-b33b-b52b255b89cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:53.210609 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed37bd0-3dbd-4b2a-b33b-b52b255b89cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:53.212325 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed37bd0-3dbd-4b2a-b33b-b52b255b89cd service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:53.223304 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1107/2260] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:14:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 17:14:53.387647 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f9616b-3915-4e9c-a9bf-596aa19aa3de service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:53.398667 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f9616b-3915-4e9c-a9bf-596aa19aa3de service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:53.406722 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a036e4-1ad8-4e36-b583-6bcfdc7de010 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:53.407400 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a036e4-1ad8-4e36-b583-6bcfdc7de010 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:53.407686 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a036e4-1ad8-4e36-b583-6bcfdc7de010 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:53.408001 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a036e4-1ad8-4e36-b583-6bcfdc7de010 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:53.468248 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a036e4-1ad8-4e36-b583-6bcfdc7de010 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:53.477943 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a036e4-1ad8-4e36-b583-6bcfdc7de010 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:53.512036 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1154/2261] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:14:53 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:14:56.206106 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e247f0-7864-46a1-895d-c2453b8d6106 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:56.206627 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e247f0-7864-46a1-895d-c2453b8d6106 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:56.206723 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e247f0-7864-46a1-895d-c2453b8d6106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:56.247339 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-05e247f0-7864-46a1-895d-c2453b8d6106 None None] MFA Rules not processed for user `8b5fdc57573c421195439a0c8e95d7d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:56.248804 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-05e247f0-7864-46a1-895d-c2453b8d6106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:56.261882 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1108/2262] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:14:59.481121 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a036e4-1ad8-4e36-b583-6bcfdc7de010 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:59.561863 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a036e4-1ad8-4e36-b583-6bcfdc7de010 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:59.567256 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b717dd0-b204-45b8-9fbf-15dbe7f00af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:59.568186 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b717dd0-b204-45b8-9fbf-15dbe7f00af0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:59.568467 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b717dd0-b204-45b8-9fbf-15dbe7f00af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:59.568793 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b717dd0-b204-45b8-9fbf-15dbe7f00af0 admin admin] PATH_INFO: `/v3/users/d221de5786e64a49b858ce7a593fee4c` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:59.570135 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b717dd0-b204-45b8-9fbf-15dbe7f00af0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d221de5786e64a49b858ce7a593fee4c)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:59.572100 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b717dd0-b204-45b8-9fbf-15dbe7f00af0 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:59.738181 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e247f0-7864-46a1-895d-c2453b8d6106 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:59.739683 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-05e247f0-7864-46a1-895d-c2453b8d6106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:59.754961 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e247f0-7864-46a1-895d-c2453b8d6106 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:14:59.763100 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51302265-9ef3-48d1-b372-419a40085f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:14:59.764818 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51302265-9ef3-48d1-b372-419a40085f73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:59.765346 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51302265-9ef3-48d1-b372-419a40085f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:59.765783 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51302265-9ef3-48d1-b372-419a40085f73 admin admin] PATH_INFO: `/v3/users/3fbc4348c7c647cc864f659e10477eab` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:59.767754 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51302265-9ef3-48d1-b372-419a40085f73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fbc4348c7c647cc864f659e10477eab)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:14:59.769993 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51302265-9ef3-48d1-b372-419a40085f73 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:14:59.800146 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7b717dd0-b204-45b8-9fbf-15dbe7f00af0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd221de5786e64a49b858ce7a593fee4c'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:59.809253 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7b717dd0-b204-45b8-9fbf-15dbe7f00af0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd221de5786e64a49b858ce7a593fee4c'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:59.810389 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7b717dd0-b204-45b8-9fbf-15dbe7f00af0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd221de5786e64a49b858ce7a593fee4c'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:59.825443 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7b717dd0-b204-45b8-9fbf-15dbe7f00af0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd221de5786e64a49b858ce7a593fee4c'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:59.827992 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:14:59.893588 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7b717dd0-b204-45b8-9fbf-15dbe7f00af0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd221de5786e64a49b858ce7a593fee4c'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:59.902933 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1155/2263] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:14:59 2022] DELETE /identity/v3/users/d221de5786e64a49b858ce7a593fee4c => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:14:59.907277 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5838f56-6f28-4ea6-96ca-e5111e2f5906 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:14:59.907574 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5838f56-6f28-4ea6-96ca-e5111e2f5906 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:14:59.907944 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5838f56-6f28-4ea6-96ca-e5111e2f5906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:14:59.925841 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-d5838f56-6f28-4ea6-96ca-e5111e2f5906 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:14:59.931190 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d5838f56-6f28-4ea6-96ca-e5111e2f5906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:14:59.985652 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-51302265-9ef3-48d1-b372-419a40085f73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fbc4348c7c647cc864f659e10477eab'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:59.987805 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1156/2264] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:14:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:14:59.997352 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5838f56-6f28-4ea6-96ca-e5111e2f5906 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:14:59.998271 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-51302265-9ef3-48d1-b372-419a40085f73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fbc4348c7c647cc864f659e10477eab'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:14:59.999443 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d5838f56-6f28-4ea6-96ca-e5111e2f5906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:00.006006 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-51302265-9ef3-48d1-b372-419a40085f73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fbc4348c7c647cc864f659e10477eab'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:00.009057 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-51302265-9ef3-48d1-b372-419a40085f73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fbc4348c7c647cc864f659e10477eab'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:00.056770 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5838f56-6f28-4ea6-96ca-e5111e2f5906 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:00.061672 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7feda68-7519-4a62-92b2-29ddbbbdd859 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:00.062638 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7feda68-7519-4a62-92b2-29ddbbbdd859 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:00.062930 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7feda68-7519-4a62-92b2-29ddbbbdd859 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:00.063231 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7feda68-7519-4a62-92b2-29ddbbbdd859 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:00.072452 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7feda68-7519-4a62-92b2-29ddbbbdd859 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:00.073879 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7feda68-7519-4a62-92b2-29ddbbbdd859 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:00.083376 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1157/2265] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:14:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 17:15:00.109253 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-51302265-9ef3-48d1-b372-419a40085f73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fbc4348c7c647cc864f659e10477eab'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:00.116029 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1109/2266] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:14:59 2022] DELETE /identity/v3/users/3fbc4348c7c647cc864f659e10477eab => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:15:00.120603 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685f38f-2435-4386-80f1-fb5461cea75e None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:00.120987 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685f38f-2435-4386-80f1-fb5461cea75e None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:00.121280 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c685f38f-2435-4386-80f1-fb5461cea75e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:00.140105 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-c685f38f-2435-4386-80f1-fb5461cea75e None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:15:00.141313 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c685f38f-2435-4386-80f1-fb5461cea75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:00.206667 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1158/2267] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:15:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:15:00.216255 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51302265-9ef3-48d1-b372-419a40085f73 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:00.218332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-51302265-9ef3-48d1-b372-419a40085f73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:00.283459 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51302265-9ef3-48d1-b372-419a40085f73 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:00.287476 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311d3a74-864d-4bbc-84d7-5a26dc426e46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:00.288311 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d3a74-864d-4bbc-84d7-5a26dc426e46 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:00.288597 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d3a74-864d-4bbc-84d7-5a26dc426e46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:00.288872 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311d3a74-864d-4bbc-84d7-5a26dc426e46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:00.300545 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311d3a74-864d-4bbc-84d7-5a26dc426e46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:00.302163 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311d3a74-864d-4bbc-84d7-5a26dc426e46 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:00.313171 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1110/2268] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:15:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 17:15:00.617359 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c685f38f-2435-4386-80f1-fb5461cea75e None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:00.619002 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c685f38f-2435-4386-80f1-fb5461cea75e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:00.634031 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c685f38f-2435-4386-80f1-fb5461cea75e None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:00.642480 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10290e9f-e85d-4f1b-bf4f-bb0046c7ca06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:00.642480 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10290e9f-e85d-4f1b-bf4f-bb0046c7ca06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:00.642480 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10290e9f-e85d-4f1b-bf4f-bb0046c7ca06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:00.642480 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10290e9f-e85d-4f1b-bf4f-bb0046c7ca06 admin admin] PATH_INFO: `/v3/projects/73dca3c38dc24c01be6a39587d9a8e44` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:00.643438 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10290e9f-e85d-4f1b-bf4f-bb0046c7ca06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73dca3c38dc24c01be6a39587d9a8e44)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:00.644369 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10290e9f-e85d-4f1b-bf4f-bb0046c7ca06 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:00.782076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:15:00.799321 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-10290e9f-e85d-4f1b-bf4f-bb0046c7ca06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73dca3c38dc24c01be6a39587d9a8e44', 'request_id': 'req-10290e9f-e85d-4f1b-bf4f-bb0046c7ca06'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:00.807341 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-10290e9f-e85d-4f1b-bf4f-bb0046c7ca06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73dca3c38dc24c01be6a39587d9a8e44', 'request_id': 'req-10290e9f-e85d-4f1b-bf4f-bb0046c7ca06'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:00.818277 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-10290e9f-e85d-4f1b-bf4f-bb0046c7ca06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73dca3c38dc24c01be6a39587d9a8e44', 'request_id': 'req-10290e9f-e85d-4f1b-bf4f-bb0046c7ca06'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:00.906328 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311d3a74-864d-4bbc-84d7-5a26dc426e46 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:00.907774 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-311d3a74-864d-4bbc-84d7-5a26dc426e46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:00.915241 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-10290e9f-e85d-4f1b-bf4f-bb0046c7ca06 admin admin] The token cache is being invalidate because project 73dca3c38dc24c01be6a39587d9a8e44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:15:00.915771 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-10290e9f-e85d-4f1b-bf4f-bb0046c7ca06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:00.916436 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-10290e9f-e85d-4f1b-bf4f-bb0046c7ca06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:00.918595 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1159/2269] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:15:00 2022] DELETE /identity/v3/projects/73dca3c38dc24c01be6a39587d9a8e44 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:15:00.923213 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10290e9f-e85d-4f1b-bf4f-bb0046c7ca06 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:00.924731 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-10290e9f-e85d-4f1b-bf4f-bb0046c7ca06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:00.931913 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311d3a74-864d-4bbc-84d7-5a26dc426e46 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:00.950166 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:15:01.018204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45485eaf-9ced-4970-846b-15e0614bc65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:01.020112 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45485eaf-9ced-4970-846b-15e0614bc65c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:01.020617 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45485eaf-9ced-4970-846b-15e0614bc65c admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:01.021135 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45485eaf-9ced-4970-846b-15e0614bc65c admin admin] PATH_INFO: `/v3/projects/f27c0fbe98e7417487064594bbac7854` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:01.024954 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45485eaf-9ced-4970-846b-15e0614bc65c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f27c0fbe98e7417487064594bbac7854)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:01.029541 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45485eaf-9ced-4970-846b-15e0614bc65c admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:01.032689 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10290e9f-e85d-4f1b-bf4f-bb0046c7ca06 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:01.038614 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2755d59-1506-4b0b-ba74-f40aae54624b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:01.040277 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2755d59-1506-4b0b-ba74-f40aae54624b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:01.040485 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2755d59-1506-4b0b-ba74-f40aae54624b admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:01.046688 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2755d59-1506-4b0b-ba74-f40aae54624b admin admin] PATH_INFO: `/v3/users/41d9e34d7689447d97ca7f33037ba711` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:01.046688 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2755d59-1506-4b0b-ba74-f40aae54624b admin admin] RBAC: Authorizing `identity:delete_user(user_id=41d9e34d7689447d97ca7f33037ba711)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:01.046688 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2755d59-1506-4b0b-ba74-f40aae54624b admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:01.129726 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-45485eaf-9ced-4970-846b-15e0614bc65c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f27c0fbe98e7417487064594bbac7854', 'request_id': 'req-45485eaf-9ced-4970-846b-15e0614bc65c'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:01.137052 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-45485eaf-9ced-4970-846b-15e0614bc65c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f27c0fbe98e7417487064594bbac7854', 'request_id': 'req-45485eaf-9ced-4970-846b-15e0614bc65c'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:01.198546 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-c2755d59-1506-4b0b-ba74-f40aae54624b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41d9e34d7689447d97ca7f33037ba711'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:01.200979 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-45485eaf-9ced-4970-846b-15e0614bc65c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f27c0fbe98e7417487064594bbac7854', 'request_id': 'req-45485eaf-9ced-4970-846b-15e0614bc65c'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:01.210017 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-c2755d59-1506-4b0b-ba74-f40aae54624b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41d9e34d7689447d97ca7f33037ba711'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:01.210818 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-c2755d59-1506-4b0b-ba74-f40aae54624b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41d9e34d7689447d97ca7f33037ba711'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:01.217576 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-45485eaf-9ced-4970-846b-15e0614bc65c admin admin] The token cache is being invalidate because project f27c0fbe98e7417487064594bbac7854 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:15:01.217996 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-45485eaf-9ced-4970-846b-15e0614bc65c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:01.218724 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-45485eaf-9ced-4970-846b-15e0614bc65c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:01.221211 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1111/2270] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:15:00 2022] DELETE /identity/v3/projects/f27c0fbe98e7417487064594bbac7854 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:15:01.228448 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-c2755d59-1506-4b0b-ba74-f40aae54624b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41d9e34d7689447d97ca7f33037ba711'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:01.228779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3798f-64f1-4d9f-bf5a-6e261b4bd40e None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:01.229694 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3798f-64f1-4d9f-bf5a-6e261b4bd40e None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:01.229694 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3798f-64f1-4d9f-bf5a-6e261b4bd40e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:01.249418 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-08e3798f-64f1-4d9f-bf5a-6e261b4bd40e None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:15:01.251307 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-08e3798f-64f1-4d9f-bf5a-6e261b4bd40e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:01.311632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1112/2271] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:15:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:15:01.318991 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec480530-c932-45ed-b8b7-4bb09fb0e125 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:01.319304 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec480530-c932-45ed-b8b7-4bb09fb0e125 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:01.319563 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec480530-c932-45ed-b8b7-4bb09fb0e125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:01.327042 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-c2755d59-1506-4b0b-ba74-f40aae54624b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41d9e34d7689447d97ca7f33037ba711'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:01.334634 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1160/2272] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:15:00 2022] DELETE /identity/v3/users/41d9e34d7689447d97ca7f33037ba711 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:15:01.341141 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-ec480530-c932-45ed-b8b7-4bb09fb0e125 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:15:01.343034 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ec480530-c932-45ed-b8b7-4bb09fb0e125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:01.403085 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1113/2273] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:15:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:15:01.407862 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2755d59-1506-4b0b-ba74-f40aae54624b admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:01.422459 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2755d59-1506-4b0b-ba74-f40aae54624b admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:01.427550 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0842e02f-7759-4b81-9925-a547383418c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:01.428281 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842e02f-7759-4b81-9925-a547383418c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:01.428564 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842e02f-7759-4b81-9925-a547383418c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:01.429439 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842e02f-7759-4b81-9925-a547383418c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:01.429562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0842e02f-7759-4b81-9925-a547383418c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:15:01.429871 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0842e02f-7759-4b81-9925-a547383418c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:01.431476 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0842e02f-7759-4b81-9925-a547383418c0 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:01.443706 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1161/2274] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:15:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:15:01.448261 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec480530-c932-45ed-b8b7-4bb09fb0e125 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:01.459370 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec480530-c932-45ed-b8b7-4bb09fb0e125 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:01.464813 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f71490b-ea6d-40f2-b5fc-7af6583754a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:01.465387 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71490b-ea6d-40f2-b5fc-7af6583754a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:01.469204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71490b-ea6d-40f2-b5fc-7af6583754a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:01.469204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f71490b-ea6d-40f2-b5fc-7af6583754a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:01.469204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f71490b-ea6d-40f2-b5fc-7af6583754a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:01.469674 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f71490b-ea6d-40f2-b5fc-7af6583754a7 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:01.603656 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:15:01.616022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1114/2275] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:15:01 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:15:01.617436 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0842e02f-7759-4b81-9925-a547383418c0 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:01.628645 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0842e02f-7759-4b81-9925-a547383418c0 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:01.653295 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:15:01.697605 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1331d7f-4e76-47dd-948f-a1dc30a663d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:01.697605 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1331d7f-4e76-47dd-948f-a1dc30a663d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:01.697605 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1331d7f-4e76-47dd-948f-a1dc30a663d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:01.697605 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1331d7f-4e76-47dd-948f-a1dc30a663d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:01.698358 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1331d7f-4e76-47dd-948f-a1dc30a663d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:01.700835 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1331d7f-4e76-47dd-948f-a1dc30a663d8 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:01.813690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1162/2276] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:15:01 2022] POST /identity/v3/users => generated 293 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:15:01.814528 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f71490b-ea6d-40f2-b5fc-7af6583754a7 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:01.825827 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f71490b-ea6d-40f2-b5fc-7af6583754a7 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:01.833483 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa5ca1-746d-482a-9d4a-f61fb9c49fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:01.833483 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa5ca1-746d-482a-9d4a-f61fb9c49fcd admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:01.833483 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa5ca1-746d-482a-9d4a-f61fb9c49fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:01.833483 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa5ca1-746d-482a-9d4a-f61fb9c49fcd admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:01.834085 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfa5ca1-746d-482a-9d4a-f61fb9c49fcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:01.835741 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfa5ca1-746d-482a-9d4a-f61fb9c49fcd admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:01.850013 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1115/2277] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:15:01.852333 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1331d7f-4e76-47dd-948f-a1dc30a663d8 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:01.862765 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1331d7f-4e76-47dd-948f-a1dc30a663d8 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:01.867561 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a80a74-5d77-4371-8242-e5495eb83389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:01.868356 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a80a74-5d77-4371-8242-e5495eb83389 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:01.868712 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a80a74-5d77-4371-8242-e5495eb83389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:01.869099 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a80a74-5d77-4371-8242-e5495eb83389 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:01.869927 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a80a74-5d77-4371-8242-e5495eb83389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:01.871641 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a80a74-5d77-4371-8242-e5495eb83389 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:01.888276 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1163/2278] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:15:01.890067 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa5ca1-746d-482a-9d4a-f61fb9c49fcd admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:01.904203 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa5ca1-746d-482a-9d4a-f61fb9c49fcd admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:01.912088 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c9f4bd-d57c-4f2b-999e-b8a9b5bfe2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:01.914761 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c9f4bd-d57c-4f2b-999e-b8a9b5bfe2cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:01.914761 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c9f4bd-d57c-4f2b-999e-b8a9b5bfe2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:01.914761 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c9f4bd-d57c-4f2b-999e-b8a9b5bfe2cb admin admin] PATH_INFO: `/v3/projects/80eb701cf6d648e5ad3235dbd00bb511/users/1c5574721f924099a7a3d84230381063/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:01.931219 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c9f4bd-d57c-4f2b-999e-b8a9b5bfe2cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=80eb701cf6d648e5ad3235dbd00bb511, user_id=1c5574721f924099a7a3d84230381063, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:01.934075 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c9f4bd-d57c-4f2b-999e-b8a9b5bfe2cb admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:02.024947 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a80a74-5d77-4371-8242-e5495eb83389 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:02.026691 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1116/2279] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:15:01 2022] PUT /identity/v3/projects/80eb701cf6d648e5ad3235dbd00bb511/users/1c5574721f924099a7a3d84230381063/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:15:02.037937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce068cd5-ce53-4206-9930-6f5d841797e1 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:02.038171 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce068cd5-ce53-4206-9930-6f5d841797e1 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:02.038787 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce068cd5-ce53-4206-9930-6f5d841797e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:02.039434 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-60a80a74-5d77-4371-8242-e5495eb83389 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:02.061526 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a80a74-5d77-4371-8242-e5495eb83389 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:02.076148 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-ce068cd5-ce53-4206-9930-6f5d841797e1 None None] MFA Rules not processed for user `1c5574721f924099a7a3d84230381063`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:15:02.092091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ce068cd5-ce53-4206-9930-6f5d841797e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:02.204686 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5dacb9-daac-47d6-9786-50a1edf89e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:02.205916 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5dacb9-daac-47d6-9786-50a1edf89e32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:02.206435 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5dacb9-daac-47d6-9786-50a1edf89e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:02.206720 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5dacb9-daac-47d6-9786-50a1edf89e32 admin admin] PATH_INFO: `/v3/projects/d1dad978a7674f3a978c55e7ad4079dc` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:02.208880 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5dacb9-daac-47d6-9786-50a1edf89e32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1dad978a7674f3a978c55e7ad4079dc)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:02.211090 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5dacb9-daac-47d6-9786-50a1edf89e32 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:02.237265 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1117/2280] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:15:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:15:02.250909 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce068cd5-ce53-4206-9930-6f5d841797e1 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:02.252615 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ce068cd5-ce53-4206-9930-6f5d841797e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:02.324216 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce068cd5-ce53-4206-9930-6f5d841797e1 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:02.327849 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2690f4-d5ab-4eaf-91a7-80e749a9c780 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:02.328504 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2690f4-d5ab-4eaf-91a7-80e749a9c780 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:02.328791 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2690f4-d5ab-4eaf-91a7-80e749a9c780 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:02.329069 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2690f4-d5ab-4eaf-91a7-80e749a9c780 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:02.338246 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2690f4-d5ab-4eaf-91a7-80e749a9c780 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:02.340612 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2690f4-d5ab-4eaf-91a7-80e749a9c780 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:02.351487 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1118/2281] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:15:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 17:15:02.356294 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-5e5dacb9-daac-47d6-9786-50a1edf89e32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1dad978a7674f3a978c55e7ad4079dc', 'request_id': 'req-5e5dacb9-daac-47d6-9786-50a1edf89e32'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:02.365268 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-5e5dacb9-daac-47d6-9786-50a1edf89e32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1dad978a7674f3a978c55e7ad4079dc', 'request_id': 'req-5e5dacb9-daac-47d6-9786-50a1edf89e32'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:02.372097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-5e5dacb9-daac-47d6-9786-50a1edf89e32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1dad978a7674f3a978c55e7ad4079dc', 'request_id': 'req-5e5dacb9-daac-47d6-9786-50a1edf89e32'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:02.478343 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-5e5dacb9-daac-47d6-9786-50a1edf89e32 admin admin] The token cache is being invalidate because project d1dad978a7674f3a978c55e7ad4079dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:15:02.479135 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-5e5dacb9-daac-47d6-9786-50a1edf89e32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:02.480071 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-5e5dacb9-daac-47d6-9786-50a1edf89e32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:02.482231 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1164/2282] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:15:02 2022] DELETE /identity/v3/projects/d1dad978a7674f3a978c55e7ad4079dc => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:15:02.487453 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf865c7-7822-4dd9-8172-9e72137fb6e1 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:02.487752 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf865c7-7822-4dd9-8172-9e72137fb6e1 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:02.488041 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf865c7-7822-4dd9-8172-9e72137fb6e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:02.514228 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-4cf865c7-7822-4dd9-8172-9e72137fb6e1 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:15:02.515605 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4cf865c7-7822-4dd9-8172-9e72137fb6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:02.526985 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:15:02.598508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1119/2283] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:15:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:15:02.606594 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41db247-c278-4926-870a-6e9849b2b73e None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:02.606891 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41db247-c278-4926-870a-6e9849b2b73e None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:02.607178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41db247-c278-4926-870a-6e9849b2b73e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:02.628768 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-e41db247-c278-4926-870a-6e9849b2b73e None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:15:02.629982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-e41db247-c278-4926-870a-6e9849b2b73e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:02.638684 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1165/2284] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:15:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:15:02.642944 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf865c7-7822-4dd9-8172-9e72137fb6e1 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:02.654118 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf865c7-7822-4dd9-8172-9e72137fb6e1 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:02.660457 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6817e3-38ae-4378-bc2c-ae99d252b82a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:02.661399 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6817e3-38ae-4378-bc2c-ae99d252b82a admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:02.661599 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6817e3-38ae-4378-bc2c-ae99d252b82a admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:02.661860 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6817e3-38ae-4378-bc2c-ae99d252b82a admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:02.662713 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6817e3-38ae-4378-bc2c-ae99d252b82a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:15:02.663282 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6817e3-38ae-4378-bc2c-ae99d252b82a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:02.664951 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6817e3-38ae-4378-bc2c-ae99d252b82a admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:02.677966 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1120/2285] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:15:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:15:02.682304 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41db247-c278-4926-870a-6e9849b2b73e None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:02.691969 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41db247-c278-4926-870a-6e9849b2b73e None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:02.696711 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda0380a-56ea-4320-845e-d6cb863d5681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:02.697369 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0380a-56ea-4320-845e-d6cb863d5681 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:02.697656 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0380a-56ea-4320-845e-d6cb863d5681 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:02.697939 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0380a-56ea-4320-845e-d6cb863d5681 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:02.698632 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda0380a-56ea-4320-845e-d6cb863d5681 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:02.700595 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda0380a-56ea-4320-845e-d6cb863d5681 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:02.772849 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1166/2286] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:15:02 2022] POST /identity/v3/projects => generated 379 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:15:02.774155 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6817e3-38ae-4378-bc2c-ae99d252b82a admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:02.801726 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6817e3-38ae-4378-bc2c-ae99d252b82a admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:02.890972 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2073e52-f2ac-4713-a113-05e4db87fcf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:02.891664 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2073e52-f2ac-4713-a113-05e4db87fcf1 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:02.892046 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2073e52-f2ac-4713-a113-05e4db87fcf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:02.892337 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2073e52-f2ac-4713-a113-05e4db87fcf1 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:02.893054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2073e52-f2ac-4713-a113-05e4db87fcf1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:02.895045 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2073e52-f2ac-4713-a113-05e4db87fcf1 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:02.993727 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1121/2287] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:15:02 2022] POST /identity/v3/users => generated 290 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:15:02.995096 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda0380a-56ea-4320-845e-d6cb863d5681 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:03.004235 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:15:03.009752 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda0380a-56ea-4320-845e-d6cb863d5681 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:03.018658 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e68640-0270-4e0f-bd0c-6ded776e724d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:03.022122 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e68640-0270-4e0f-bd0c-6ded776e724d admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:03.022122 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e68640-0270-4e0f-bd0c-6ded776e724d admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:03.022122 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e68640-0270-4e0f-bd0c-6ded776e724d admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:03.022122 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e68640-0270-4e0f-bd0c-6ded776e724d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:03.023343 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e68640-0270-4e0f-bd0c-6ded776e724d admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:03.036308 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1167/2288] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:15:03.040328 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2073e52-f2ac-4713-a113-05e4db87fcf1 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:03.053137 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2073e52-f2ac-4713-a113-05e4db87fcf1 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:03.058272 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3043a41a-16af-432a-97c8-fe4e1175056a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:03.059023 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3043a41a-16af-432a-97c8-fe4e1175056a admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:03.059317 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3043a41a-16af-432a-97c8-fe4e1175056a admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:03.059595 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3043a41a-16af-432a-97c8-fe4e1175056a admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:03.060387 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3043a41a-16af-432a-97c8-fe4e1175056a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:03.062779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3043a41a-16af-432a-97c8-fe4e1175056a admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:03.076651 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1122/2289] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:15:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:15:03.078023 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e68640-0270-4e0f-bd0c-6ded776e724d admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:03.096378 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e68640-0270-4e0f-bd0c-6ded776e724d admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:03.106357 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763296e8-e7e6-4313-881a-007350c0f720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:03.107275 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763296e8-e7e6-4313-881a-007350c0f720 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:03.108379 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763296e8-e7e6-4313-881a-007350c0f720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:03.108668 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763296e8-e7e6-4313-881a-007350c0f720 admin admin] PATH_INFO: `/v3/projects/45f5b2e49fd34460b3d3fc129f61a02e/users/6948c3b9666e457f9f3297b6a66b64fa/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:03.127579 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763296e8-e7e6-4313-881a-007350c0f720 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45f5b2e49fd34460b3d3fc129f61a02e, user_id=6948c3b9666e457f9f3297b6a66b64fa, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:03.130557 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763296e8-e7e6-4313-881a-007350c0f720 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:03.208558 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1168/2290] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:15:03 2022] PUT /identity/v3/projects/45f5b2e49fd34460b3d3fc129f61a02e/users/6948c3b9666e457f9f3297b6a66b64fa/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:15:03.219948 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e876b-34ac-48d4-9254-cf34ae0a9b7a None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:03.220221 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e876b-34ac-48d4-9254-cf34ae0a9b7a None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:03.220675 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14e876b-34ac-48d4-9254-cf34ae0a9b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:03.256000 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-a14e876b-34ac-48d4-9254-cf34ae0a9b7a None None] MFA Rules not processed for user `6948c3b9666e457f9f3297b6a66b64fa`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:15:03.258010 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-a14e876b-34ac-48d4-9254-cf34ae0a9b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:03.366104 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1123/2291] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:15:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:15:03.370470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763296e8-e7e6-4313-881a-007350c0f720 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:03.470507 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763296e8-e7e6-4313-881a-007350c0f720 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:03.475374 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff31853-b4e0-46cc-9e5c-06f3538024f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:03.476085 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff31853-b4e0-46cc-9e5c-06f3538024f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:03.476416 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff31853-b4e0-46cc-9e5c-06f3538024f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:03.476808 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff31853-b4e0-46cc-9e5c-06f3538024f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:03.478199 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff31853-b4e0-46cc-9e5c-06f3538024f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:03.480257 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff31853-b4e0-46cc-9e5c-06f3538024f9 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:03.578687 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1169/2292] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:15:03 2022] POST /identity/v3/projects => generated 379 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:15:03.584270 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e876b-34ac-48d4-9254-cf34ae0a9b7a None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:03.595903 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14e876b-34ac-48d4-9254-cf34ae0a9b7a None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:03.606888 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:15:03.655703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a300ed9-f84b-40f6-84f2-5ff1f8e6d73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:03.656406 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a300ed9-f84b-40f6-84f2-5ff1f8e6d73c admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:03.656727 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a300ed9-f84b-40f6-84f2-5ff1f8e6d73c admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:03.657018 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a300ed9-f84b-40f6-84f2-5ff1f8e6d73c admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:03.658055 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a300ed9-f84b-40f6-84f2-5ff1f8e6d73c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:03.660854 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a300ed9-f84b-40f6-84f2-5ff1f8e6d73c admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:03.772399 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1124/2293] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:15:03 2022] POST /identity/v3/users => generated 290 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:15:03.780423 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff31853-b4e0-46cc-9e5c-06f3538024f9 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:03.796689 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff31853-b4e0-46cc-9e5c-06f3538024f9 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:03.803539 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c19c55-a587-47e3-b6b6-e825f4ce92b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:03.804171 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c19c55-a587-47e3-b6b6-e825f4ce92b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:03.804453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c19c55-a587-47e3-b6b6-e825f4ce92b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:03.804728 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c19c55-a587-47e3-b6b6-e825f4ce92b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:03.805455 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c19c55-a587-47e3-b6b6-e825f4ce92b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:03.807039 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c19c55-a587-47e3-b6b6-e825f4ce92b9 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:03.817758 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1170/2294] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:15:03 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:15:03.821926 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a300ed9-f84b-40f6-84f2-5ff1f8e6d73c admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:03.833101 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a300ed9-f84b-40f6-84f2-5ff1f8e6d73c admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:03.838076 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0b6e1a-cc0a-410d-8564-f6a23c55cf77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:03.838810 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b6e1a-cc0a-410d-8564-f6a23c55cf77 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:03.839206 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b6e1a-cc0a-410d-8564-f6a23c55cf77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:03.839488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b6e1a-cc0a-410d-8564-f6a23c55cf77 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:03.840273 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0b6e1a-cc0a-410d-8564-f6a23c55cf77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:03.841983 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0b6e1a-cc0a-410d-8564-f6a23c55cf77 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:03.853111 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1125/2295] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:15:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:15:03.857167 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c19c55-a587-47e3-b6b6-e825f4ce92b9 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:03.868637 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c19c55-a587-47e3-b6b6-e825f4ce92b9 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:03.877171 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09663add-ac77-4456-9c2d-1da6dd16352c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:03.878070 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09663add-ac77-4456-9c2d-1da6dd16352c admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:03.878374 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09663add-ac77-4456-9c2d-1da6dd16352c admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:03.878648 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09663add-ac77-4456-9c2d-1da6dd16352c admin admin] PATH_INFO: `/v3/projects/c82d16b966a24f5b868bf7a6b23bf87a/users/1d87de467171469096ee660fda0ec099/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:03.894319 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09663add-ac77-4456-9c2d-1da6dd16352c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c82d16b966a24f5b868bf7a6b23bf87a, user_id=1d87de467171469096ee660fda0ec099, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:03.897024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09663add-ac77-4456-9c2d-1da6dd16352c admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:04.013646 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1171/2296] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:15:03 2022] PUT /identity/v3/projects/c82d16b966a24f5b868bf7a6b23bf87a/users/1d87de467171469096ee660fda0ec099/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:15:04.024690 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32499352-c0b8-4b51-a7f9-37fab6303027 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:04.025184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32499352-c0b8-4b51-a7f9-37fab6303027 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:04.025272 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32499352-c0b8-4b51-a7f9-37fab6303027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:04.052439 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-32499352-c0b8-4b51-a7f9-37fab6303027 None None] MFA Rules not processed for user `1d87de467171469096ee660fda0ec099`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:15:04.053709 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-32499352-c0b8-4b51-a7f9-37fab6303027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:04.122542 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1126/2297] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:15:04 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:15:04.128260 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09663add-ac77-4456-9c2d-1da6dd16352c admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:04.192335 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09663add-ac77-4456-9c2d-1da6dd16352c admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:04.197616 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177bc3d7-ca63-4b7e-a1c6-ee611be465b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:04.198284 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177bc3d7-ca63-4b7e-a1c6-ee611be465b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:04.199679 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177bc3d7-ca63-4b7e-a1c6-ee611be465b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:04.199679 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177bc3d7-ca63-4b7e-a1c6-ee611be465b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:04.199679 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177bc3d7-ca63-4b7e-a1c6-ee611be465b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:04.202075 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177bc3d7-ca63-4b7e-a1c6-ee611be465b8 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:04.317371 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1172/2298] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:15:04 2022] POST /identity/v3/projects => generated 379 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:15:04.321408 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32499352-c0b8-4b51-a7f9-37fab6303027 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:04.332844 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32499352-c0b8-4b51-a7f9-37fab6303027 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:04.387939 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f42cb8-dad8-4084-a4a0-ad44c7ee0620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:04.388610 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f42cb8-dad8-4084-a4a0-ad44c7ee0620 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:04.389507 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f42cb8-dad8-4084-a4a0-ad44c7ee0620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:04.389789 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f42cb8-dad8-4084-a4a0-ad44c7ee0620 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:04.390542 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f42cb8-dad8-4084-a4a0-ad44c7ee0620 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:04.392507 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f42cb8-dad8-4084-a4a0-ad44c7ee0620 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:04.524261 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1127/2299] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:15:04 2022] POST /identity/v3/users => generated 296 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:15:04.528126 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177bc3d7-ca63-4b7e-a1c6-ee611be465b8 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:04.536274 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:15:04.541430 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177bc3d7-ca63-4b7e-a1c6-ee611be465b8 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:04.548415 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c40ea0-ab80-44cf-84da-84c5f7fd0be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:04.549496 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c40ea0-ab80-44cf-84da-84c5f7fd0be2 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:04.549851 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c40ea0-ab80-44cf-84da-84c5f7fd0be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:04.550311 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c40ea0-ab80-44cf-84da-84c5f7fd0be2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:04.551125 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c40ea0-ab80-44cf-84da-84c5f7fd0be2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:04.553171 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c40ea0-ab80-44cf-84da-84c5f7fd0be2 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:04.571999 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1173/2300] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:15:04 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:15:04.580401 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f42cb8-dad8-4084-a4a0-ad44c7ee0620 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:04.594583 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f42cb8-dad8-4084-a4a0-ad44c7ee0620 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:04.599566 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352f454-044a-47c9-9b60-4bf9f8e1eda4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:04.600494 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352f454-044a-47c9-9b60-4bf9f8e1eda4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:04.600759 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352f454-044a-47c9-9b60-4bf9f8e1eda4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:04.601019 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352f454-044a-47c9-9b60-4bf9f8e1eda4 admin admin] PATH_INFO: `/v3/projects/e83224467dee4acea193556d28cc02ac/users/9adef490e7ec437aa329f259aa863322/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:04.604248 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:15:04.617427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0352f454-044a-47c9-9b60-4bf9f8e1eda4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e83224467dee4acea193556d28cc02ac, user_id=9adef490e7ec437aa329f259aa863322, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:04.620113 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0352f454-044a-47c9-9b60-4bf9f8e1eda4 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:04.693040 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1128/2301] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:15:04 2022] PUT /identity/v3/projects/e83224467dee4acea193556d28cc02ac/users/9adef490e7ec437aa329f259aa863322/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:15:04.697256 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c40ea0-ab80-44cf-84da-84c5f7fd0be2 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:04.708217 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c40ea0-ab80-44cf-84da-84c5f7fd0be2 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:04.773945 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a25cbc1-ad1e-46de-a0d2-4a80e229401e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:04.774633 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a25cbc1-ad1e-46de-a0d2-4a80e229401e admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:04.777261 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a25cbc1-ad1e-46de-a0d2-4a80e229401e admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:04.777261 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a25cbc1-ad1e-46de-a0d2-4a80e229401e admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:04.777261 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a25cbc1-ad1e-46de-a0d2-4a80e229401e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:04.778699 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a25cbc1-ad1e-46de-a0d2-4a80e229401e admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:04.791067 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1174/2302] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:15:04 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:15:04.796619 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352f454-044a-47c9-9b60-4bf9f8e1eda4 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:04.808900 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352f454-044a-47c9-9b60-4bf9f8e1eda4 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:04.814167 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac42d4e-33e6-444c-8c9d-4525393a206d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:04.815053 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac42d4e-33e6-444c-8c9d-4525393a206d admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:04.815335 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac42d4e-33e6-444c-8c9d-4525393a206d admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:04.815609 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac42d4e-33e6-444c-8c9d-4525393a206d admin admin] PATH_INFO: `/v3/projects/e83224467dee4acea193556d28cc02ac/users/9adef490e7ec437aa329f259aa863322/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:04.817628 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac42d4e-33e6-444c-8c9d-4525393a206d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e83224467dee4acea193556d28cc02ac, user_id=9adef490e7ec437aa329f259aa863322, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:04.824542 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac42d4e-33e6-444c-8c9d-4525393a206d admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:04.832315 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1129/2303] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:15:04 2022] PUT /identity/v3/projects/e83224467dee4acea193556d28cc02ac/users/9adef490e7ec437aa329f259aa863322/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:15:04.843039 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870c2cb-6373-4a12-8877-03a851746dde None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:04.843436 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870c2cb-6373-4a12-8877-03a851746dde None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:04.843747 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2870c2cb-6373-4a12-8877-03a851746dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:04.870327 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-2870c2cb-6373-4a12-8877-03a851746dde None None] MFA Rules not processed for user `9adef490e7ec437aa329f259aa863322`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:15:04.871579 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-2870c2cb-6373-4a12-8877-03a851746dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:04.955019 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1175/2304] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:15:04 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:15:04.968368 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac42d4e-33e6-444c-8c9d-4525393a206d admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:04.970168 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-8ac42d4e-33e6-444c-8c9d-4525393a206d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:05.026718 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac42d4e-33e6-444c-8c9d-4525393a206d admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:05.030977 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba0bf0-ab96-4f00-88a5-c6459940fb3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:05.031609 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba0bf0-ab96-4f00-88a5-c6459940fb3f service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:05.031921 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba0bf0-ab96-4f00-88a5-c6459940fb3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:05.032213 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ba0bf0-ab96-4f00-88a5-c6459940fb3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:05.042092 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ba0bf0-ab96-4f00-88a5-c6459940fb3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:05.043551 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ba0bf0-ab96-4f00-88a5-c6459940fb3f service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:05.058179 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1130/2305] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:15:04 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:15:05.187137 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2870c2cb-6373-4a12-8877-03a851746dde None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:05.198677 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2870c2cb-6373-4a12-8877-03a851746dde None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:05.204246 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e876e70-c5b9-4c6e-b653-7a7b0ed58eae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:05.205134 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e876e70-c5b9-4c6e-b653-7a7b0ed58eae service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:05.205769 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e876e70-c5b9-4c6e-b653-7a7b0ed58eae service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:05.206071 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e876e70-c5b9-4c6e-b653-7a7b0ed58eae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:05.267607 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e876e70-c5b9-4c6e-b653-7a7b0ed58eae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:05.269059 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e876e70-c5b9-4c6e-b653-7a7b0ed58eae service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:05.278168 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1176/2306] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:15:05 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:15:05.590889 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba0bf0-ab96-4f00-88a5-c6459940fb3f service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:05.592347 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-76ba0bf0-ab96-4f00-88a5-c6459940fb3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:05.674246 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ba0bf0-ab96-4f00-88a5-c6459940fb3f service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:05.679880 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5991b-8746-4ae2-91a7-8c19bcb34fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:05.680845 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5991b-8746-4ae2-91a7-8c19bcb34fb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:05.681128 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5991b-8746-4ae2-91a7-8c19bcb34fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:05.681522 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e5991b-8746-4ae2-91a7-8c19bcb34fb6 admin admin] PATH_INFO: `/v3/users/1c5574721f924099a7a3d84230381063` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:05.683105 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e5991b-8746-4ae2-91a7-8c19bcb34fb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c5574721f924099a7a3d84230381063)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:05.684694 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e5991b-8746-4ae2-91a7-8c19bcb34fb6 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:05.886463 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-35e5991b-8746-4ae2-91a7-8c19bcb34fb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c5574721f924099a7a3d84230381063'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:05.901511 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-35e5991b-8746-4ae2-91a7-8c19bcb34fb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c5574721f924099a7a3d84230381063'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:05.908585 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-35e5991b-8746-4ae2-91a7-8c19bcb34fb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c5574721f924099a7a3d84230381063'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:05.909290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-35e5991b-8746-4ae2-91a7-8c19bcb34fb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c5574721f924099a7a3d84230381063'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:05.911866 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:15:05.979145 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-35e5991b-8746-4ae2-91a7-8c19bcb34fb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c5574721f924099a7a3d84230381063'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:05.987092 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1131/2307] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:15:05 2022] DELETE /identity/v3/users/1c5574721f924099a7a3d84230381063 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:15:05.991528 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f3f8b-5778-4390-8103-d67c0c814bd0 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:05.991976 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f3f8b-5778-4390-8103-d67c0c814bd0 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:05.992142 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f3f8b-5778-4390-8103-d67c0c814bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:06.010262 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-ed3f3f8b-5778-4390-8103-d67c0c814bd0 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:15:06.011466 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-ed3f3f8b-5778-4390-8103-d67c0c814bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:06.019520 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:15:06.071607 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1177/2308] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:15:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:15:06.081482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5991b-8746-4ae2-91a7-8c19bcb34fb6 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:06.082971 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-35e5991b-8746-4ae2-91a7-8c19bcb34fb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:06.140077 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e5991b-8746-4ae2-91a7-8c19bcb34fb6 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:06.143675 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c6009b-8105-4957-9151-71edb64c4f3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:06.148809 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c6009b-8105-4957-9151-71edb64c4f3a service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:06.148809 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c6009b-8105-4957-9151-71edb64c4f3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:06.148809 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c6009b-8105-4957-9151-71edb64c4f3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:06.154187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c6009b-8105-4957-9151-71edb64c4f3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:06.156176 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c6009b-8105-4957-9151-71edb64c4f3a service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:06.166407 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1132/2309] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:15:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:15:06.549959 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f3f8b-5778-4390-8103-d67c0c814bd0 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:06.551396 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-ed3f3f8b-5778-4390-8103-d67c0c814bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:06.564378 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f3f8b-5778-4390-8103-d67c0c814bd0 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:06.569377 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22d41cb-84de-4f02-91b4-2dd59117a922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:06.570245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d41cb-84de-4f02-91b4-2dd59117a922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:06.570525 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d41cb-84de-4f02-91b4-2dd59117a922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:06.570798 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22d41cb-84de-4f02-91b4-2dd59117a922 admin admin] PATH_INFO: `/v3/projects/80eb701cf6d648e5ad3235dbd00bb511` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:06.572325 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22d41cb-84de-4f02-91b4-2dd59117a922 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80eb701cf6d648e5ad3235dbd00bb511)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:06.574337 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22d41cb-84de-4f02-91b4-2dd59117a922 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:06.705885 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e22d41cb-84de-4f02-91b4-2dd59117a922 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80eb701cf6d648e5ad3235dbd00bb511', 'request_id': 'req-e22d41cb-84de-4f02-91b4-2dd59117a922'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:06.712175 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e22d41cb-84de-4f02-91b4-2dd59117a922 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80eb701cf6d648e5ad3235dbd00bb511', 'request_id': 'req-e22d41cb-84de-4f02-91b4-2dd59117a922'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:06.724171 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e22d41cb-84de-4f02-91b4-2dd59117a922 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80eb701cf6d648e5ad3235dbd00bb511', 'request_id': 'req-e22d41cb-84de-4f02-91b4-2dd59117a922'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:06.829566 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e22d41cb-84de-4f02-91b4-2dd59117a922 admin admin] The token cache is being invalidate because project 80eb701cf6d648e5ad3235dbd00bb511 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:15:06.829865 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e22d41cb-84de-4f02-91b4-2dd59117a922 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:06.830526 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-e22d41cb-84de-4f02-91b4-2dd59117a922 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:06.837849 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1178/2310] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:15:06 2022] DELETE /identity/v3/projects/80eb701cf6d648e5ad3235dbd00bb511 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:15:09.336424 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c6009b-8105-4957-9151-71edb64c4f3a service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:09.338190 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-77c6009b-8105-4957-9151-71edb64c4f3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:09.353155 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c6009b-8105-4957-9151-71edb64c4f3a service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:09.419429 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d3a599-e113-4550-becf-cce649c766b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:09.420354 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3a599-e113-4550-becf-cce649c766b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:09.420645 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3a599-e113-4550-becf-cce649c766b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:09.421010 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d3a599-e113-4550-becf-cce649c766b0 admin admin] PATH_INFO: `/v3/users/6948c3b9666e457f9f3297b6a66b64fa` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:09.422800 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d3a599-e113-4550-becf-cce649c766b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6948c3b9666e457f9f3297b6a66b64fa)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:09.424779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d3a599-e113-4550-becf-cce649c766b0 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:09.619820 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-71d3a599-e113-4550-becf-cce649c766b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6948c3b9666e457f9f3297b6a66b64fa'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:09.631552 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-71d3a599-e113-4550-becf-cce649c766b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6948c3b9666e457f9f3297b6a66b64fa'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:09.638594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-71d3a599-e113-4550-becf-cce649c766b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6948c3b9666e457f9f3297b6a66b64fa'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:09.639253 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-71d3a599-e113-4550-becf-cce649c766b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6948c3b9666e457f9f3297b6a66b64fa'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:09.720958 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-71d3a599-e113-4550-becf-cce649c766b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6948c3b9666e457f9f3297b6a66b64fa'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:09.727605 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1133/2311] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:15:09 2022] DELETE /identity/v3/users/6948c3b9666e457f9f3297b6a66b64fa => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:15:09.732252 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a74ede-a784-43a3-b793-f31eef29e354 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:09.732744 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a74ede-a784-43a3-b793-f31eef29e354 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:09.733003 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a74ede-a784-43a3-b793-f31eef29e354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:09.750835 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-03a74ede-a784-43a3-b793-f31eef29e354 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:15:09.752127 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-03a74ede-a784-43a3-b793-f31eef29e354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:09.815184 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1179/2312] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:15:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:15:09.824695 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d3a599-e113-4550-becf-cce649c766b0 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:09.826085 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-71d3a599-e113-4550-becf-cce649c766b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:09.845601 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:15:09.883603 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d3a599-e113-4550-becf-cce649c766b0 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:09.887160 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd383c5-9bb9-4c0d-b049-91b26baf9438 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:09.887821 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd383c5-9bb9-4c0d-b049-91b26baf9438 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:09.888103 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd383c5-9bb9-4c0d-b049-91b26baf9438 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:09.888379 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd383c5-9bb9-4c0d-b049-91b26baf9438 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:09.897749 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd383c5-9bb9-4c0d-b049-91b26baf9438 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:09.899170 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd383c5-9bb9-4c0d-b049-91b26baf9438 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:09.909953 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1134/2313] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:15:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:15:10.304930 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a74ede-a784-43a3-b793-f31eef29e354 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:10.306353 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-03a74ede-a784-43a3-b793-f31eef29e354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:10.318645 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a74ede-a784-43a3-b793-f31eef29e354 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:10.323868 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b85aa-5791-48fc-89d3-cfb5bce21eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:10.324736 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b85aa-5791-48fc-89d3-cfb5bce21eeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:10.325097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b85aa-5791-48fc-89d3-cfb5bce21eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:10.325402 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482b85aa-5791-48fc-89d3-cfb5bce21eeb admin admin] PATH_INFO: `/v3/projects/45f5b2e49fd34460b3d3fc129f61a02e` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:10.326585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482b85aa-5791-48fc-89d3-cfb5bce21eeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=45f5b2e49fd34460b3d3fc129f61a02e)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:10.329117 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482b85aa-5791-48fc-89d3-cfb5bce21eeb admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:10.418308 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:15:10.429110 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-482b85aa-5791-48fc-89d3-cfb5bce21eeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45f5b2e49fd34460b3d3fc129f61a02e', 'request_id': 'req-482b85aa-5791-48fc-89d3-cfb5bce21eeb'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:10.436197 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-482b85aa-5791-48fc-89d3-cfb5bce21eeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45f5b2e49fd34460b3d3fc129f61a02e', 'request_id': 'req-482b85aa-5791-48fc-89d3-cfb5bce21eeb'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:10.443270 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-482b85aa-5791-48fc-89d3-cfb5bce21eeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45f5b2e49fd34460b3d3fc129f61a02e', 'request_id': 'req-482b85aa-5791-48fc-89d3-cfb5bce21eeb'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:10.498877 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-482b85aa-5791-48fc-89d3-cfb5bce21eeb admin admin] The token cache is being invalidate because project 45f5b2e49fd34460b3d3fc129f61a02e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:15:10.499935 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-482b85aa-5791-48fc-89d3-cfb5bce21eeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:10.500573 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-482b85aa-5791-48fc-89d3-cfb5bce21eeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:10.505918 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1180/2314] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:15:10 2022] DELETE /identity/v3/projects/45f5b2e49fd34460b3d3fc129f61a02e => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:15:10.506863 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd383c5-9bb9-4c0d-b049-91b26baf9438 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:10.517504 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd383c5-9bb9-4c0d-b049-91b26baf9438 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:10.590828 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aba489-1695-414e-bb7e-d3a61f5faa0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:10.591719 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aba489-1695-414e-bb7e-d3a61f5faa0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:10.592034 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aba489-1695-414e-bb7e-d3a61f5faa0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:10.592318 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aba489-1695-414e-bb7e-d3a61f5faa0a admin admin] PATH_INFO: `/v3/users/1d87de467171469096ee660fda0ec099` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:10.593557 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aba489-1695-414e-bb7e-d3a61f5faa0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d87de467171469096ee660fda0ec099)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:10.595482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aba489-1695-414e-bb7e-d3a61f5faa0a admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:10.770384 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-83aba489-1695-414e-bb7e-d3a61f5faa0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d87de467171469096ee660fda0ec099'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:10.783089 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-83aba489-1695-414e-bb7e-d3a61f5faa0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d87de467171469096ee660fda0ec099'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:10.789884 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-83aba489-1695-414e-bb7e-d3a61f5faa0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d87de467171469096ee660fda0ec099'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:10.790564 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-83aba489-1695-414e-bb7e-d3a61f5faa0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d87de467171469096ee660fda0ec099'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:10.840433 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-83aba489-1695-414e-bb7e-d3a61f5faa0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d87de467171469096ee660fda0ec099'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:10.846483 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1135/2315] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:15:10 2022] DELETE /identity/v3/users/1d87de467171469096ee660fda0ec099 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:15:11.213339 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b85aa-5791-48fc-89d3-cfb5bce21eeb admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:11.214849 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-482b85aa-5791-48fc-89d3-cfb5bce21eeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:11.228093 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482b85aa-5791-48fc-89d3-cfb5bce21eeb admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:11.284720 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672d81f-588a-4f6c-9d38-3c261de8b69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:11.285567 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672d81f-588a-4f6c-9d38-3c261de8b69d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:11.285846 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672d81f-588a-4f6c-9d38-3c261de8b69d admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:11.286120 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d672d81f-588a-4f6c-9d38-3c261de8b69d admin admin] PATH_INFO: `/v3/projects/c82d16b966a24f5b868bf7a6b23bf87a` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:11.287297 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d672d81f-588a-4f6c-9d38-3c261de8b69d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c82d16b966a24f5b868bf7a6b23bf87a)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:11.289255 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d672d81f-588a-4f6c-9d38-3c261de8b69d admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:11.399518 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-d672d81f-588a-4f6c-9d38-3c261de8b69d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c82d16b966a24f5b868bf7a6b23bf87a', 'request_id': 'req-d672d81f-588a-4f6c-9d38-3c261de8b69d'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:11.405864 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-d672d81f-588a-4f6c-9d38-3c261de8b69d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c82d16b966a24f5b868bf7a6b23bf87a', 'request_id': 'req-d672d81f-588a-4f6c-9d38-3c261de8b69d'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:11.411476 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-d672d81f-588a-4f6c-9d38-3c261de8b69d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c82d16b966a24f5b868bf7a6b23bf87a', 'request_id': 'req-d672d81f-588a-4f6c-9d38-3c261de8b69d'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:11.508331 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-d672d81f-588a-4f6c-9d38-3c261de8b69d admin admin] The token cache is being invalidate because project c82d16b966a24f5b868bf7a6b23bf87a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:15:11.508558 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-d672d81f-588a-4f6c-9d38-3c261de8b69d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:11.509209 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-d672d81f-588a-4f6c-9d38-3c261de8b69d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:11.510760 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1181/2316] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:15:11 2022] DELETE /identity/v3/projects/c82d16b966a24f5b868bf7a6b23bf87a => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:15:11.514875 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aba489-1695-414e-bb7e-d3a61f5faa0a admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:11.516384 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-83aba489-1695-414e-bb7e-d3a61f5faa0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:11.529424 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aba489-1695-414e-bb7e-d3a61f5faa0a admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:11.563031 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:15:11.585798 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56893d8d-c8e7-4525-9306-074e284bea41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:11.586681 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56893d8d-c8e7-4525-9306-074e284bea41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:11.586971 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56893d8d-c8e7-4525-9306-074e284bea41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:11.587370 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56893d8d-c8e7-4525-9306-074e284bea41 admin admin] PATH_INFO: `/v3/users/9adef490e7ec437aa329f259aa863322` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:11.588614 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56893d8d-c8e7-4525-9306-074e284bea41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9adef490e7ec437aa329f259aa863322)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:11.590567 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56893d8d-c8e7-4525-9306-074e284bea41 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:11.781960 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-56893d8d-c8e7-4525-9306-074e284bea41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9adef490e7ec437aa329f259aa863322'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:11.795491 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-56893d8d-c8e7-4525-9306-074e284bea41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9adef490e7ec437aa329f259aa863322'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:11.802941 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-56893d8d-c8e7-4525-9306-074e284bea41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9adef490e7ec437aa329f259aa863322'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:11.803656 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-56893d8d-c8e7-4525-9306-074e284bea41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9adef490e7ec437aa329f259aa863322'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:11.852095 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-56893d8d-c8e7-4525-9306-074e284bea41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9adef490e7ec437aa329f259aa863322'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:11.858337 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1136/2317] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:15:11 2022] DELETE /identity/v3/users/9adef490e7ec437aa329f259aa863322 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:15:12.270990 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672d81f-588a-4f6c-9d38-3c261de8b69d admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:12.272657 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d672d81f-588a-4f6c-9d38-3c261de8b69d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:12.286435 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d672d81f-588a-4f6c-9d38-3c261de8b69d admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:12.334034 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:15:12.343591 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205f0d49-b5af-4465-a2ee-bb8a18833101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:12.344479 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f0d49-b5af-4465-a2ee-bb8a18833101 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:12.344764 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f0d49-b5af-4465-a2ee-bb8a18833101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:12.345040 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205f0d49-b5af-4465-a2ee-bb8a18833101 admin admin] PATH_INFO: `/v3/projects/e83224467dee4acea193556d28cc02ac` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:12.346242 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205f0d49-b5af-4465-a2ee-bb8a18833101 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e83224467dee4acea193556d28cc02ac)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:12.348288 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205f0d49-b5af-4465-a2ee-bb8a18833101 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:12.447344 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-205f0d49-b5af-4465-a2ee-bb8a18833101 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e83224467dee4acea193556d28cc02ac', 'request_id': 'req-205f0d49-b5af-4465-a2ee-bb8a18833101'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:12.454016 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-205f0d49-b5af-4465-a2ee-bb8a18833101 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e83224467dee4acea193556d28cc02ac', 'request_id': 'req-205f0d49-b5af-4465-a2ee-bb8a18833101'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:12.459471 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-205f0d49-b5af-4465-a2ee-bb8a18833101 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e83224467dee4acea193556d28cc02ac', 'request_id': 'req-205f0d49-b5af-4465-a2ee-bb8a18833101'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:12.554545 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-205f0d49-b5af-4465-a2ee-bb8a18833101 admin admin] The token cache is being invalidate because project e83224467dee4acea193556d28cc02ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:15:12.554741 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-205f0d49-b5af-4465-a2ee-bb8a18833101 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:12.555350 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-205f0d49-b5af-4465-a2ee-bb8a18833101 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:12.556967 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1182/2318] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:15:12 2022] DELETE /identity/v3/projects/e83224467dee4acea193556d28cc02ac => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:15:12.563373 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e72374-2be0-48eb-99e4-5f2f7ec3145f None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:12.563852 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e72374-2be0-48eb-99e4-5f2f7ec3145f None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:12.563940 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e72374-2be0-48eb-99e4-5f2f7ec3145f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:12.582343 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-c3e72374-2be0-48eb-99e4-5f2f7ec3145f None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:15:12.583527 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c3e72374-2be0-48eb-99e4-5f2f7ec3145f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:12.641110 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1137/2319] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:15:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:15:12.649448 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4fddcd-b5d5-49d4-981d-c106a1731ba0 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:12.649748 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4fddcd-b5d5-49d4-981d-c106a1731ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:12.650028 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4fddcd-b5d5-49d4-981d-c106a1731ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:12.668147 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-fe4fddcd-b5d5-49d4-981d-c106a1731ba0 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:15:12.669325 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-fe4fddcd-b5d5-49d4-981d-c106a1731ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:12.677638 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1183/2320] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:15:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:15:12.681516 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e72374-2be0-48eb-99e4-5f2f7ec3145f None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:12.691518 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e72374-2be0-48eb-99e4-5f2f7ec3145f None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:12.696056 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fb8f87-ea03-4af6-840f-f786a559e9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:12.696704 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb8f87-ea03-4af6-840f-f786a559e9d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:12.696989 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb8f87-ea03-4af6-840f-f786a559e9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:12.697266 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb8f87-ea03-4af6-840f-f786a559e9d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:12.697990 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fb8f87-ea03-4af6-840f-f786a559e9d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:15:12.698298 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fb8f87-ea03-4af6-840f-f786a559e9d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:12.699935 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fb8f87-ea03-4af6-840f-f786a559e9d5 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:12.711139 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1138/2321] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:15:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:15:12.715328 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4fddcd-b5d5-49d4-981d-c106a1731ba0 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:12.724655 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4fddcd-b5d5-49d4-981d-c106a1731ba0 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:12.729197 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e354e4c-4193-47a4-a5eb-0f2fa2b32e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:12.729842 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e354e4c-4193-47a4-a5eb-0f2fa2b32e19 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:12.730126 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e354e4c-4193-47a4-a5eb-0f2fa2b32e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:12.730400 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e354e4c-4193-47a4-a5eb-0f2fa2b32e19 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:12.731095 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e354e4c-4193-47a4-a5eb-0f2fa2b32e19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:12.733045 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e354e4c-4193-47a4-a5eb-0f2fa2b32e19 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:12.831836 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1184/2322] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:15:12 2022] POST /identity/v3/projects => generated 365 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:15:12.836294 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fb8f87-ea03-4af6-840f-f786a559e9d5 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:12.857618 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fb8f87-ea03-4af6-840f-f786a559e9d5 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:12.913750 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:15:12.933460 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b9257e-aa28-4443-be57-4718a406fc0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:12.934200 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9257e-aa28-4443-be57-4718a406fc0b admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:12.934492 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9257e-aa28-4443-be57-4718a406fc0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:12.934779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b9257e-aa28-4443-be57-4718a406fc0b admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:12.935496 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b9257e-aa28-4443-be57-4718a406fc0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:12.938745 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b9257e-aa28-4443-be57-4718a406fc0b admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:13.018420 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1139/2323] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:15:12 2022] POST /identity/v3/users => generated 283 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:15:13.022474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e354e4c-4193-47a4-a5eb-0f2fa2b32e19 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:13.034180 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e354e4c-4193-47a4-a5eb-0f2fa2b32e19 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:13.038770 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4b346-e6e3-4184-a8d6-4a877cc0bd50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:13.041395 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4b346-e6e3-4184-a8d6-4a877cc0bd50 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:13.041395 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4b346-e6e3-4184-a8d6-4a877cc0bd50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:13.041395 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4b346-e6e3-4184-a8d6-4a877cc0bd50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:13.041395 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f4b346-e6e3-4184-a8d6-4a877cc0bd50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:13.042375 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f4b346-e6e3-4184-a8d6-4a877cc0bd50 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:13.056133 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1185/2324] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:15:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:15:13.060460 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b9257e-aa28-4443-be57-4718a406fc0b admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:13.070681 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b9257e-aa28-4443-be57-4718a406fc0b admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:13.075445 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b381f07-481b-4ae5-b9e8-8e986dee7a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:13.076138 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b381f07-481b-4ae5-b9e8-8e986dee7a41 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:13.076426 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b381f07-481b-4ae5-b9e8-8e986dee7a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:13.076704 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b381f07-481b-4ae5-b9e8-8e986dee7a41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:13.077468 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b381f07-481b-4ae5-b9e8-8e986dee7a41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:13.079070 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b381f07-481b-4ae5-b9e8-8e986dee7a41 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:13.089964 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1140/2325] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:15:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:15:13.094735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4b346-e6e3-4184-a8d6-4a877cc0bd50 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:13.109555 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4b346-e6e3-4184-a8d6-4a877cc0bd50 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:13.114312 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2d49e8-6ca7-4fe6-afc4-2e03008a3b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:13.115195 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d49e8-6ca7-4fe6-afc4-2e03008a3b48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:13.115481 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d49e8-6ca7-4fe6-afc4-2e03008a3b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:13.115791 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2d49e8-6ca7-4fe6-afc4-2e03008a3b48 admin admin] PATH_INFO: `/v3/projects/f14b44324d854d4f8c61351bc43ab781/users/b56588f745c94909acc940e933594bf1/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:13.131683 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2d49e8-6ca7-4fe6-afc4-2e03008a3b48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f14b44324d854d4f8c61351bc43ab781, user_id=b56588f745c94909acc940e933594bf1, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:13.134595 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2d49e8-6ca7-4fe6-afc4-2e03008a3b48 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:13.254400 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1186/2326] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:15:13 2022] PUT /identity/v3/projects/f14b44324d854d4f8c61351bc43ab781/users/b56588f745c94909acc940e933594bf1/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:15:13.266763 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8a9e2-7ab2-4c57-886a-8b5630294a36 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:13.267056 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8a9e2-7ab2-4c57-886a-8b5630294a36 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:13.267336 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8a9e2-7ab2-4c57-886a-8b5630294a36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:13.293731 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-39e8a9e2-7ab2-4c57-886a-8b5630294a36 None None] MFA Rules not processed for user `b56588f745c94909acc940e933594bf1`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:15:13.294977 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-39e8a9e2-7ab2-4c57-886a-8b5630294a36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:13.357189 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1141/2327] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:15:13 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:15:13.362006 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2d49e8-6ca7-4fe6-afc4-2e03008a3b48 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:13.424154 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2d49e8-6ca7-4fe6-afc4-2e03008a3b48 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:13.428830 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e82c1f-53ea-4a29-b659-aa92eb1f864c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:13.429486 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e82c1f-53ea-4a29-b659-aa92eb1f864c admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:13.429770 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e82c1f-53ea-4a29-b659-aa92eb1f864c admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:13.430046 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e82c1f-53ea-4a29-b659-aa92eb1f864c admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:13.430744 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e82c1f-53ea-4a29-b659-aa92eb1f864c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:13.432688 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e82c1f-53ea-4a29-b659-aa92eb1f864c admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:13.532317 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1187/2328] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:15:13 2022] POST /identity/v3/projects => generated 365 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:15:13.536993 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8a9e2-7ab2-4c57-886a-8b5630294a36 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:13.547721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e8a9e2-7ab2-4c57-886a-8b5630294a36 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:13.603599 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e224716-339a-4439-bca7-c322725235e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:13.604401 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e224716-339a-4439-bca7-c322725235e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:13.604497 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e224716-339a-4439-bca7-c322725235e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:13.604768 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e224716-339a-4439-bca7-c322725235e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:13.605449 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e224716-339a-4439-bca7-c322725235e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:13.607387 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e224716-339a-4439-bca7-c322725235e0 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:13.728225 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1142/2329] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:15:13 2022] POST /identity/v3/users => generated 289 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:15:13.731933 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e82c1f-53ea-4a29-b659-aa92eb1f864c admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:13.737946 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:15:13.745530 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e82c1f-53ea-4a29-b659-aa92eb1f864c admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:13.750487 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02708e38-91b6-4e5e-b4af-f61d187d34b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:13.751687 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02708e38-91b6-4e5e-b4af-f61d187d34b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:13.752021 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02708e38-91b6-4e5e-b4af-f61d187d34b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:13.752294 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02708e38-91b6-4e5e-b4af-f61d187d34b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:13.753033 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02708e38-91b6-4e5e-b4af-f61d187d34b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:13.754623 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02708e38-91b6-4e5e-b4af-f61d187d34b6 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:13.765353 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1188/2330] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:15:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:15:13.770122 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e224716-339a-4439-bca7-c322725235e0 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:13.781588 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e224716-339a-4439-bca7-c322725235e0 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:13.786386 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef1200-f26b-49b0-9376-f453c77a21ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:13.787332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef1200-f26b-49b0-9376-f453c77a21ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:13.787760 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef1200-f26b-49b0-9376-f453c77a21ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:13.787926 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ef1200-f26b-49b0-9376-f453c77a21ff admin admin] PATH_INFO: `/v3/projects/2293106436ea4aceb3e9521ea5a656ae/users/3aa341315b724e4d92ff742db43009c3/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:13.808368 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef1200-f26b-49b0-9376-f453c77a21ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=2293106436ea4aceb3e9521ea5a656ae, user_id=3aa341315b724e4d92ff742db43009c3, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:13.811598 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ef1200-f26b-49b0-9376-f453c77a21ff admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:13.882419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1143/2331] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:15:13 2022] PUT /identity/v3/projects/2293106436ea4aceb3e9521ea5a656ae/users/3aa341315b724e4d92ff742db43009c3/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:15:13.886356 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02708e38-91b6-4e5e-b4af-f61d187d34b6 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:13.896783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02708e38-91b6-4e5e-b4af-f61d187d34b6 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:13.975556 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a012734b-d4da-4652-8308-c5d965c94082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:13.976427 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012734b-d4da-4652-8308-c5d965c94082 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:13.977201 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012734b-d4da-4652-8308-c5d965c94082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:13.977201 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a012734b-d4da-4652-8308-c5d965c94082 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:13.978313 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a012734b-d4da-4652-8308-c5d965c94082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:13.980113 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a012734b-d4da-4652-8308-c5d965c94082 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:13.993106 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1189/2332] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:15:13 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:15:13.997436 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef1200-f26b-49b0-9376-f453c77a21ff admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:14.012510 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ef1200-f26b-49b0-9376-f453c77a21ff admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:14.020844 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d2c1b-475a-4cf7-86cf-5e332e9ae740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:14.021825 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d2c1b-475a-4cf7-86cf-5e332e9ae740 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:14.022109 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d2c1b-475a-4cf7-86cf-5e332e9ae740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:14.022518 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28d2c1b-475a-4cf7-86cf-5e332e9ae740 admin admin] PATH_INFO: `/v3/projects/2293106436ea4aceb3e9521ea5a656ae/users/3aa341315b724e4d92ff742db43009c3/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:14.025052 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d2c1b-475a-4cf7-86cf-5e332e9ae740 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2293106436ea4aceb3e9521ea5a656ae, user_id=3aa341315b724e4d92ff742db43009c3, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:14.029065 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28d2c1b-475a-4cf7-86cf-5e332e9ae740 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:14.040674 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1144/2333] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:15:13 2022] PUT /identity/v3/projects/2293106436ea4aceb3e9521ea5a656ae/users/3aa341315b724e4d92ff742db43009c3/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:15:14.116300 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aef750-e8a7-4d53-afa2-03f976d28ae6 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:14.116594 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aef750-e8a7-4d53-afa2-03f976d28ae6 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:14.116876 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aef750-e8a7-4d53-afa2-03f976d28ae6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:14.150702 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-d5aef750-e8a7-4d53-afa2-03f976d28ae6 None None] MFA Rules not processed for user `3aa341315b724e4d92ff742db43009c3`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:15:14.152298 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d5aef750-e8a7-4d53-afa2-03f976d28ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:14.240691 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1190/2334] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:15:14 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:15:14.254641 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d2c1b-475a-4cf7-86cf-5e332e9ae740 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:14.257248 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-a28d2c1b-475a-4cf7-86cf-5e332e9ae740 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:14.273865 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:15:14.331173 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28d2c1b-475a-4cf7-86cf-5e332e9ae740 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:14.335014 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a2beba-ae14-4111-a915-49525726d681 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:14.335669 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a2beba-ae14-4111-a915-49525726d681 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:14.336170 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a2beba-ae14-4111-a915-49525726d681 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:14.336459 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a2beba-ae14-4111-a915-49525726d681 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:14.346983 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a2beba-ae14-4111-a915-49525726d681 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:14.348545 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a2beba-ae14-4111-a915-49525726d681 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:14.359335 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1145/2335] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:15:14 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:15:14.583257 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aef750-e8a7-4d53-afa2-03f976d28ae6 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:14.584694 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d5aef750-e8a7-4d53-afa2-03f976d28ae6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:14.597307 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aef750-e8a7-4d53-afa2-03f976d28ae6 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:14.600886 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b228906-248a-46f5-9881-ed73ed68ecfa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:14.603817 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b228906-248a-46f5-9881-ed73ed68ecfa service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:14.603817 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b228906-248a-46f5-9881-ed73ed68ecfa service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:14.603817 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b228906-248a-46f5-9881-ed73ed68ecfa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:14.659561 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b228906-248a-46f5-9881-ed73ed68ecfa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:14.660964 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b228906-248a-46f5-9881-ed73ed68ecfa service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:14.670102 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1191/2336] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:15:14 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:15:19.391708 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a2beba-ae14-4111-a915-49525726d681 service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:19.393870 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-10a2beba-ae14-4111-a915-49525726d681 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:19.545939 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a2beba-ae14-4111-a915-49525726d681 service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:19.551889 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15915ec-a9d7-4e7c-8bbe-46b4738afcca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:19.554321 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15915ec-a9d7-4e7c-8bbe-46b4738afcca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:19.555026 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15915ec-a9d7-4e7c-8bbe-46b4738afcca admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:19.555484 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15915ec-a9d7-4e7c-8bbe-46b4738afcca admin admin] PATH_INFO: `/v3/users/2851b96fac754276b541b529cfbc59df` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:19.557072 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15915ec-a9d7-4e7c-8bbe-46b4738afcca admin admin] RBAC: Authorizing `identity:delete_user(user_id=2851b96fac754276b541b529cfbc59df)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:19.559643 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15915ec-a9d7-4e7c-8bbe-46b4738afcca admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:19.758554 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d15915ec-a9d7-4e7c-8bbe-46b4738afcca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2851b96fac754276b541b529cfbc59df'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:19.779498 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d15915ec-a9d7-4e7c-8bbe-46b4738afcca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2851b96fac754276b541b529cfbc59df'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:19.788032 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d15915ec-a9d7-4e7c-8bbe-46b4738afcca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2851b96fac754276b541b529cfbc59df'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:19.788779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d15915ec-a9d7-4e7c-8bbe-46b4738afcca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2851b96fac754276b541b529cfbc59df'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:19.869550 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d15915ec-a9d7-4e7c-8bbe-46b4738afcca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2851b96fac754276b541b529cfbc59df'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:19.881902 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1146/2337] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:15:19 2022] DELETE /identity/v3/users/2851b96fac754276b541b529cfbc59df => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:15:19.885022 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175464b9-897c-4129-a7f8-8a5f28c8287a None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:19.886031 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175464b9-897c-4129-a7f8-8a5f28c8287a None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:19.886500 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175464b9-897c-4129-a7f8-8a5f28c8287a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:19.907675 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-175464b9-897c-4129-a7f8-8a5f28c8287a None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:15:19.909002 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-175464b9-897c-4129-a7f8-8a5f28c8287a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:19.914992 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:15:19.976804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1192/2338] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:15:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:15:19.986488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15915ec-a9d7-4e7c-8bbe-46b4738afcca admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:19.988481 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d15915ec-a9d7-4e7c-8bbe-46b4738afcca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:20.018927 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:15:20.044677 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15915ec-a9d7-4e7c-8bbe-46b4738afcca admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:20.048393 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbab85c2-9ec0-4866-9526-8acec80e3a54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:20.049038 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab85c2-9ec0-4866-9526-8acec80e3a54 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:20.049324 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab85c2-9ec0-4866-9526-8acec80e3a54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:20.050243 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbab85c2-9ec0-4866-9526-8acec80e3a54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:20.060041 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab85c2-9ec0-4866-9526-8acec80e3a54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:20.061499 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbab85c2-9ec0-4866-9526-8acec80e3a54 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:20.071188 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1147/2339] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:15:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:15:20.513927 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175464b9-897c-4129-a7f8-8a5f28c8287a None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:20.515394 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-175464b9-897c-4129-a7f8-8a5f28c8287a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:20.530186 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175464b9-897c-4129-a7f8-8a5f28c8287a None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:20.536787 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5b4085-842d-47ac-9814-3fc5d801a6e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:20.539875 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b4085-842d-47ac-9814-3fc5d801a6e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:20.539875 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b4085-842d-47ac-9814-3fc5d801a6e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:20.539875 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5b4085-842d-47ac-9814-3fc5d801a6e1 admin admin] PATH_INFO: `/v3/projects/9c32202a246c4bada91bcd9459df925d` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:20.539875 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5b4085-842d-47ac-9814-3fc5d801a6e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c32202a246c4bada91bcd9459df925d)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:20.541316 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5b4085-842d-47ac-9814-3fc5d801a6e1 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:20.657357 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8d5b4085-842d-47ac-9814-3fc5d801a6e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c32202a246c4bada91bcd9459df925d', 'request_id': 'req-8d5b4085-842d-47ac-9814-3fc5d801a6e1'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:20.663557 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8d5b4085-842d-47ac-9814-3fc5d801a6e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c32202a246c4bada91bcd9459df925d', 'request_id': 'req-8d5b4085-842d-47ac-9814-3fc5d801a6e1'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:20.669156 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8d5b4085-842d-47ac-9814-3fc5d801a6e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c32202a246c4bada91bcd9459df925d', 'request_id': 'req-8d5b4085-842d-47ac-9814-3fc5d801a6e1'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:20.753331 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8d5b4085-842d-47ac-9814-3fc5d801a6e1 admin admin] The token cache is being invalidate because project 9c32202a246c4bada91bcd9459df925d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:15:20.753754 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8d5b4085-842d-47ac-9814-3fc5d801a6e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:20.754942 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8d5b4085-842d-47ac-9814-3fc5d801a6e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:20.756785 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1193/2340] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:15:20 2022] DELETE /identity/v3/projects/9c32202a246c4bada91bcd9459df925d => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:15:20.761938 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21b531-5f69-4dbd-9c1a-8f3df6e6c287 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:20.762398 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21b531-5f69-4dbd-9c1a-8f3df6e6c287 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:20.763409 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21b531-5f69-4dbd-9c1a-8f3df6e6c287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:20.782830 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-cd21b531-5f69-4dbd-9c1a-8f3df6e6c287 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:15:20.784054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-cd21b531-5f69-4dbd-9c1a-8f3df6e6c287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:20.849540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1148/2341] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:15:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:15:20.857069 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630de7d-8fb0-4604-a4d0-ceb92411ee56 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:20.857363 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630de7d-8fb0-4604-a4d0-ceb92411ee56 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:20.857642 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630de7d-8fb0-4604-a4d0-ceb92411ee56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:20.876302 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-f630de7d-8fb0-4604-a4d0-ceb92411ee56 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:15:20.877507 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f630de7d-8fb0-4604-a4d0-ceb92411ee56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:20.885945 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1194/2342] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:15:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:15:20.890009 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21b531-5f69-4dbd-9c1a-8f3df6e6c287 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:20.899556 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21b531-5f69-4dbd-9c1a-8f3df6e6c287 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:20.904554 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d001e7f-1ec9-4523-a007-538ed611d670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:20.905206 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d001e7f-1ec9-4523-a007-538ed611d670 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:20.905497 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d001e7f-1ec9-4523-a007-538ed611d670 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:20.905778 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d001e7f-1ec9-4523-a007-538ed611d670 admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:20.906511 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d001e7f-1ec9-4523-a007-538ed611d670 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:15:20.906823 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d001e7f-1ec9-4523-a007-538ed611d670 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:20.908469 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d001e7f-1ec9-4523-a007-538ed611d670 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:20.919810 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1149/2343] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:15:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:15:20.924005 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f630de7d-8fb0-4604-a4d0-ceb92411ee56 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:20.933594 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f630de7d-8fb0-4604-a4d0-ceb92411ee56 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:20.938218 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cbc8e7-ffca-4ea7-a87e-f407b3951e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:20.938854 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cbc8e7-ffca-4ea7-a87e-f407b3951e7f admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:20.939139 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cbc8e7-ffca-4ea7-a87e-f407b3951e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:20.939417 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cbc8e7-ffca-4ea7-a87e-f407b3951e7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:20.940141 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cbc8e7-ffca-4ea7-a87e-f407b3951e7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:20.942621 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3cbc8e7-ffca-4ea7-a87e-f407b3951e7f admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:21.046349 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1195/2344] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:15:20 2022] POST /identity/v3/projects => generated 373 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:15:21.050444 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d001e7f-1ec9-4523-a007-538ed611d670 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:21.061007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d001e7f-1ec9-4523-a007-538ed611d670 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:21.118839 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7186f41f-09a2-4c3b-beca-f59a54a9703e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:21.119522 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186f41f-09a2-4c3b-beca-f59a54a9703e admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:21.119845 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186f41f-09a2-4c3b-beca-f59a54a9703e admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:21.120125 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7186f41f-09a2-4c3b-beca-f59a54a9703e admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:21.121657 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7186f41f-09a2-4c3b-beca-f59a54a9703e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:21.122736 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7186f41f-09a2-4c3b-beca-f59a54a9703e admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:21.200688 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1150/2345] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:15:21 2022] POST /identity/v3/users => generated 287 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:15:21.204725 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cbc8e7-ffca-4ea7-a87e-f407b3951e7f admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:21.215071 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3cbc8e7-ffca-4ea7-a87e-f407b3951e7f admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:21.219913 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd839a31-23fc-4252-b30f-35c27021608c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:21.220568 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd839a31-23fc-4252-b30f-35c27021608c admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:21.220855 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd839a31-23fc-4252-b30f-35c27021608c admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:21.221132 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd839a31-23fc-4252-b30f-35c27021608c admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:21.221991 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd839a31-23fc-4252-b30f-35c27021608c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:21.224146 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd839a31-23fc-4252-b30f-35c27021608c admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:21.236141 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1196/2346] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:15:21.241376 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7186f41f-09a2-4c3b-beca-f59a54a9703e admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:21.251155 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7186f41f-09a2-4c3b-beca-f59a54a9703e admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:21.255951 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf08da-50f9-4dea-b063-bc4eaa31aa30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:21.256670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf08da-50f9-4dea-b063-bc4eaa31aa30 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:21.261719 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf08da-50f9-4dea-b063-bc4eaa31aa30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:21.261719 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baf08da-50f9-4dea-b063-bc4eaa31aa30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:21.261719 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baf08da-50f9-4dea-b063-bc4eaa31aa30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:21.261719 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baf08da-50f9-4dea-b063-bc4eaa31aa30 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:21.275179 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1151/2347] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:15:21.276175 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd839a31-23fc-4252-b30f-35c27021608c admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:21.280864 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf08da-50f9-4dea-b063-bc4eaa31aa30 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:21.284865 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-0baf08da-50f9-4dea-b063-bc4eaa31aa30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:21.288153 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd839a31-23fc-4252-b30f-35c27021608c admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:21.292893 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3344960b-e7b4-4bdd-8996-d90c495530bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:21.293784 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344960b-e7b4-4bdd-8996-d90c495530bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:21.294067 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344960b-e7b4-4bdd-8996-d90c495530bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:21.294343 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344960b-e7b4-4bdd-8996-d90c495530bf admin admin] PATH_INFO: `/v3/projects/a98ed6fa4ade48e4a66cb733c2cb76d8/users/49ee4818fef74e70bd1a352ab9c86e33/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:21.297805 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baf08da-50f9-4dea-b063-bc4eaa31aa30 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:21.303152 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faf13ac-eea1-4b24-92a0-91afe005e0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:21.304071 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf13ac-eea1-4b24-92a0-91afe005e0f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:21.307691 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf13ac-eea1-4b24-92a0-91afe005e0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:21.307691 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faf13ac-eea1-4b24-92a0-91afe005e0f7 admin admin] PATH_INFO: `/v3/users/b56588f745c94909acc940e933594bf1` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:21.307691 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faf13ac-eea1-4b24-92a0-91afe005e0f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b56588f745c94909acc940e933594bf1)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:21.308121 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faf13ac-eea1-4b24-92a0-91afe005e0f7 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:21.310334 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3344960b-e7b4-4bdd-8996-d90c495530bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=a98ed6fa4ade48e4a66cb733c2cb76d8, user_id=49ee4818fef74e70bd1a352ab9c86e33, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:21.312984 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3344960b-e7b4-4bdd-8996-d90c495530bf admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:21.393550 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1197/2348] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:15:21 2022] PUT /identity/v3/projects/a98ed6fa4ade48e4a66cb733c2cb76d8/users/49ee4818fef74e70bd1a352ab9c86e33/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:15:21.405959 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735bb728-3ec6-4acb-b690-0c3c8e4b8694 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:21.406272 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735bb728-3ec6-4acb-b690-0c3c8e4b8694 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:21.406554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735bb728-3ec6-4acb-b690-0c3c8e4b8694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:21.435634 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-735bb728-3ec6-4acb-b690-0c3c8e4b8694 None None] MFA Rules not processed for user `49ee4818fef74e70bd1a352ab9c86e33`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:15:21.437295 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-735bb728-3ec6-4acb-b690-0c3c8e4b8694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:21.456936 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:15:21.509492 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1198/2349] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:15:21 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:15:21.520950 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735bb728-3ec6-4acb-b690-0c3c8e4b8694 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:21.522357 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-735bb728-3ec6-4acb-b690-0c3c8e4b8694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:21.544480 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:15:21.558157 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6faf13ac-eea1-4b24-92a0-91afe005e0f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b56588f745c94909acc940e933594bf1'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:21.570789 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6faf13ac-eea1-4b24-92a0-91afe005e0f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b56588f745c94909acc940e933594bf1'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:21.580041 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6faf13ac-eea1-4b24-92a0-91afe005e0f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b56588f745c94909acc940e933594bf1'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:21.580400 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735bb728-3ec6-4acb-b690-0c3c8e4b8694 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:21.580868 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6faf13ac-eea1-4b24-92a0-91afe005e0f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b56588f745c94909acc940e933594bf1'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:21.585063 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82797a5d-c463-4362-ad0f-6d6f8a0c0c06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:21.585726 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82797a5d-c463-4362-ad0f-6d6f8a0c0c06 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:21.586008 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82797a5d-c463-4362-ad0f-6d6f8a0c0c06 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:21.586285 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82797a5d-c463-4362-ad0f-6d6f8a0c0c06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:21.595362 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82797a5d-c463-4362-ad0f-6d6f8a0c0c06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:21.597002 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82797a5d-c463-4362-ad0f-6d6f8a0c0c06 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:21.608581 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1199/2350] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:15:21 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 17:15:21.640781 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6faf13ac-eea1-4b24-92a0-91afe005e0f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b56588f745c94909acc940e933594bf1'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:21.649324 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1152/2351] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:15:21 2022] DELETE /identity/v3/users/b56588f745c94909acc940e933594bf1 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:15:21.654305 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4af90-7752-48c3-b360-082e5bacbfbf None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:21.654606 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4af90-7752-48c3-b360-082e5bacbfbf None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:21.654884 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4af90-7752-48c3-b360-082e5bacbfbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:21.675025 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-7db4af90-7752-48c3-b360-082e5bacbfbf None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:15:21.680682 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-7db4af90-7752-48c3-b360-082e5bacbfbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:21.761100 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1200/2352] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:15:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:15:21.786173 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faf13ac-eea1-4b24-92a0-91afe005e0f7 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:21.786173 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-6faf13ac-eea1-4b24-92a0-91afe005e0f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:21.857424 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faf13ac-eea1-4b24-92a0-91afe005e0f7 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:21.861465 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b28edc-c1a0-42b6-a901-d9c97a5b610c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:21.862127 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b28edc-c1a0-42b6-a901-d9c97a5b610c service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:21.862469 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b28edc-c1a0-42b6-a901-d9c97a5b610c service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:21.862753 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b28edc-c1a0-42b6-a901-d9c97a5b610c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:21.876623 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b28edc-c1a0-42b6-a901-d9c97a5b610c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:21.878085 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b28edc-c1a0-42b6-a901-d9c97a5b610c service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:21.889261 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1153/2353] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:15:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:15:22.431204 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4af90-7752-48c3-b360-082e5bacbfbf None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:22.432628 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-7db4af90-7752-48c3-b360-082e5bacbfbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:22.445500 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4af90-7752-48c3-b360-082e5bacbfbf None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:22.450728 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35348649-0bc3-43dd-b12f-cfa02530f0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:22.462498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35348649-0bc3-43dd-b12f-cfa02530f0d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:22.462498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35348649-0bc3-43dd-b12f-cfa02530f0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:22.462498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35348649-0bc3-43dd-b12f-cfa02530f0d8 admin admin] PATH_INFO: `/v3/projects/f14b44324d854d4f8c61351bc43ab781` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:22.462498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35348649-0bc3-43dd-b12f-cfa02530f0d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f14b44324d854d4f8c61351bc43ab781)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:22.462498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35348649-0bc3-43dd-b12f-cfa02530f0d8 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:22.551465 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-35348649-0bc3-43dd-b12f-cfa02530f0d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f14b44324d854d4f8c61351bc43ab781', 'request_id': 'req-35348649-0bc3-43dd-b12f-cfa02530f0d8'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:22.557604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-35348649-0bc3-43dd-b12f-cfa02530f0d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f14b44324d854d4f8c61351bc43ab781', 'request_id': 'req-35348649-0bc3-43dd-b12f-cfa02530f0d8'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:22.561396 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:15:22.563587 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-35348649-0bc3-43dd-b12f-cfa02530f0d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f14b44324d854d4f8c61351bc43ab781', 'request_id': 'req-35348649-0bc3-43dd-b12f-cfa02530f0d8'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:22.612490 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-35348649-0bc3-43dd-b12f-cfa02530f0d8 admin admin] The token cache is being invalidate because project f14b44324d854d4f8c61351bc43ab781 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:15:22.612784 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-35348649-0bc3-43dd-b12f-cfa02530f0d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:22.613952 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-35348649-0bc3-43dd-b12f-cfa02530f0d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:22.615822 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1201/2354] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:15:22 2022] DELETE /identity/v3/projects/f14b44324d854d4f8c61351bc43ab781 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:15:22.619686 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b28edc-c1a0-42b6-a901-d9c97a5b610c service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:22.630414 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b28edc-c1a0-42b6-a901-d9c97a5b610c service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:22.691995 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff86cc4f-7768-438b-b65d-c949a6b33b0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:22.692978 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86cc4f-7768-438b-b65d-c949a6b33b0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:22.693267 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86cc4f-7768-438b-b65d-c949a6b33b0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:22.693548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86cc4f-7768-438b-b65d-c949a6b33b0b admin admin] PATH_INFO: `/v3/users/3aa341315b724e4d92ff742db43009c3` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:22.694768 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff86cc4f-7768-438b-b65d-c949a6b33b0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3aa341315b724e4d92ff742db43009c3)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:22.696777 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff86cc4f-7768-438b-b65d-c949a6b33b0b admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:22.843294 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ff86cc4f-7768-438b-b65d-c949a6b33b0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3aa341315b724e4d92ff742db43009c3'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:22.848319 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:15:22.856347 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ff86cc4f-7768-438b-b65d-c949a6b33b0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3aa341315b724e4d92ff742db43009c3'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:22.863158 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ff86cc4f-7768-438b-b65d-c949a6b33b0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3aa341315b724e4d92ff742db43009c3'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:22.864443 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ff86cc4f-7768-438b-b65d-c949a6b33b0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3aa341315b724e4d92ff742db43009c3'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:22.936440 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ff86cc4f-7768-438b-b65d-c949a6b33b0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3aa341315b724e4d92ff742db43009c3'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:22.942713 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1154/2355] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:15:22 2022] DELETE /identity/v3/users/3aa341315b724e4d92ff742db43009c3 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:15:23.628774 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35348649-0bc3-43dd-b12f-cfa02530f0d8 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:23.631319 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-35348649-0bc3-43dd-b12f-cfa02530f0d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:23.639938 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff86cc4f-7768-438b-b65d-c949a6b33b0b admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:23.641422 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ff86cc4f-7768-438b-b65d-c949a6b33b0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:23.712533 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35348649-0bc3-43dd-b12f-cfa02530f0d8 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:23.765951 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff86cc4f-7768-438b-b65d-c949a6b33b0b admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:23.771291 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb5c535-2e34-45f0-bcee-59b5a1f5f415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:23.772369 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb5c535-2e34-45f0-bcee-59b5a1f5f415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:23.773559 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb5c535-2e34-45f0-bcee-59b5a1f5f415 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:23.773559 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb5c535-2e34-45f0-bcee-59b5a1f5f415 admin admin] PATH_INFO: `/v3/projects/2293106436ea4aceb3e9521ea5a656ae` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:23.774287 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb5c535-2e34-45f0-bcee-59b5a1f5f415 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2293106436ea4aceb3e9521ea5a656ae)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:23.776893 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb5c535-2e34-45f0-bcee-59b5a1f5f415 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:23.790126 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95d948b-d483-4b63-98de-f594c51f566f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:23.791050 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d948b-d483-4b63-98de-f594c51f566f service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:23.791339 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d948b-d483-4b63-98de-f594c51f566f service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:23.791621 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95d948b-d483-4b63-98de-f594c51f566f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:23.793602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c95d948b-d483-4b63-98de-f594c51f566f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:23.856011 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95d948b-d483-4b63-98de-f594c51f566f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:23.857435 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95d948b-d483-4b63-98de-f594c51f566f service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:23.867812 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1202/2356] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:15:23 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:15:23.921114 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ffb5c535-2e34-45f0-bcee-59b5a1f5f415 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2293106436ea4aceb3e9521ea5a656ae', 'request_id': 'req-ffb5c535-2e34-45f0-bcee-59b5a1f5f415'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:23.927424 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ffb5c535-2e34-45f0-bcee-59b5a1f5f415 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2293106436ea4aceb3e9521ea5a656ae', 'request_id': 'req-ffb5c535-2e34-45f0-bcee-59b5a1f5f415'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:24.062554 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ffb5c535-2e34-45f0-bcee-59b5a1f5f415 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2293106436ea4aceb3e9521ea5a656ae', 'request_id': 'req-ffb5c535-2e34-45f0-bcee-59b5a1f5f415'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:24.077929 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ffb5c535-2e34-45f0-bcee-59b5a1f5f415 admin admin] The token cache is being invalidate because project 2293106436ea4aceb3e9521ea5a656ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:15:24.078267 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ffb5c535-2e34-45f0-bcee-59b5a1f5f415 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:24.078976 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ffb5c535-2e34-45f0-bcee-59b5a1f5f415 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:24.081157 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1155/2357] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:15:23 2022] DELETE /identity/v3/projects/2293106436ea4aceb3e9521ea5a656ae => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:15:24.086530 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae02f6-40e8-41c5-aac7-fa806c44977b None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:24.086827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae02f6-40e8-41c5-aac7-fa806c44977b None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:24.087107 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ae02f6-40e8-41c5-aac7-fa806c44977b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:24.110559 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-71ae02f6-40e8-41c5-aac7-fa806c44977b None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:15:24.111791 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-71ae02f6-40e8-41c5-aac7-fa806c44977b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:24.174868 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:15:24.217268 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1203/2358] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:15:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:15:24.224103 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e1e66-edc1-44a4-88ae-8ca406069115 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:24.224844 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e1e66-edc1-44a4-88ae-8ca406069115 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:24.225366 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e1e66-edc1-44a4-88ae-8ca406069115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:24.272257 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-dd6e1e66-edc1-44a4-88ae-8ca406069115 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:15:24.274100 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-dd6e1e66-edc1-44a4-88ae-8ca406069115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:24.283645 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1156/2359] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:15:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:15:24.288002 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae02f6-40e8-41c5-aac7-fa806c44977b None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:24.300274 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ae02f6-40e8-41c5-aac7-fa806c44977b None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:24.306093 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3046bd-93ca-414a-b842-765e5e236fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:24.306943 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3046bd-93ca-414a-b842-765e5e236fcf admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:24.307329 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3046bd-93ca-414a-b842-765e5e236fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:24.307709 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3046bd-93ca-414a-b842-765e5e236fcf admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:24.308851 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3046bd-93ca-414a-b842-765e5e236fcf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:15:24.309268 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3046bd-93ca-414a-b842-765e5e236fcf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:24.311801 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3046bd-93ca-414a-b842-765e5e236fcf admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:24.337126 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1204/2360] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:15:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:15:24.341452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e1e66-edc1-44a4-88ae-8ca406069115 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:24.351950 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e1e66-edc1-44a4-88ae-8ca406069115 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:24.363358 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c65e1-13e7-401b-aa5f-5216882e4b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:24.363358 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c65e1-13e7-401b-aa5f-5216882e4b97 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:24.363358 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c65e1-13e7-401b-aa5f-5216882e4b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:24.363358 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2c65e1-13e7-401b-aa5f-5216882e4b97 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:24.364363 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c65e1-13e7-401b-aa5f-5216882e4b97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:24.366486 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2c65e1-13e7-401b-aa5f-5216882e4b97 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:24.530062 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1157/2361] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:15:24 2022] POST /identity/v3/projects => generated 377 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:15:24.534311 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3046bd-93ca-414a-b842-765e5e236fcf admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:24.545548 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3046bd-93ca-414a-b842-765e5e236fcf admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:24.616225 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30ed0ce-a73f-4e95-9cb2-82c15f5f6348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:24.617215 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ed0ce-a73f-4e95-9cb2-82c15f5f6348 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:24.617581 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ed0ce-a73f-4e95-9cb2-82c15f5f6348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:24.617909 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30ed0ce-a73f-4e95-9cb2-82c15f5f6348 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:24.618864 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30ed0ce-a73f-4e95-9cb2-82c15f5f6348 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:24.621278 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30ed0ce-a73f-4e95-9cb2-82c15f5f6348 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:24.706851 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1205/2362] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:15:24 2022] POST /identity/v3/users => generated 289 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:15:24.709342 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c65e1-13e7-401b-aa5f-5216882e4b97 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:24.724958 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2c65e1-13e7-401b-aa5f-5216882e4b97 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:24.734140 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15c587-ea8e-433d-801a-8600198d403a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:24.734990 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15c587-ea8e-433d-801a-8600198d403a admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:24.735380 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15c587-ea8e-433d-801a-8600198d403a admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:24.735803 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a15c587-ea8e-433d-801a-8600198d403a admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:24.741710 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a15c587-ea8e-433d-801a-8600198d403a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:24.743513 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a15c587-ea8e-433d-801a-8600198d403a admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:24.755317 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1158/2363] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:15:24.759392 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30ed0ce-a73f-4e95-9cb2-82c15f5f6348 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:24.771541 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30ed0ce-a73f-4e95-9cb2-82c15f5f6348 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:24.777209 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b87bc0c-4820-42de-a94d-6c960842a4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:24.778126 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b87bc0c-4820-42de-a94d-6c960842a4dc admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:24.778634 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b87bc0c-4820-42de-a94d-6c960842a4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:24.779278 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b87bc0c-4820-42de-a94d-6c960842a4dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:24.780471 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b87bc0c-4820-42de-a94d-6c960842a4dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:24.782958 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b87bc0c-4820-42de-a94d-6c960842a4dc admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:24.797241 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1206/2364] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:15:24.798648 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15c587-ea8e-433d-801a-8600198d403a admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:24.809641 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a15c587-ea8e-433d-801a-8600198d403a admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:24.814703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c582fa2-3204-44a6-a575-4365987099ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:24.815623 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c582fa2-3204-44a6-a575-4365987099ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:24.815936 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c582fa2-3204-44a6-a575-4365987099ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:24.816475 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c582fa2-3204-44a6-a575-4365987099ae admin admin] PATH_INFO: `/v3/projects/7654ee6379f14e95aacd02ddb886378b/users/b9ee1a98b5d541dfb6ab76f440cd43e2/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:24.835895 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c582fa2-3204-44a6-a575-4365987099ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=7654ee6379f14e95aacd02ddb886378b, user_id=b9ee1a98b5d541dfb6ab76f440cd43e2, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:24.838720 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c582fa2-3204-44a6-a575-4365987099ae admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:24.842921 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:15:24.915007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1159/2365] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:15:24 2022] PUT /identity/v3/projects/7654ee6379f14e95aacd02ddb886378b/users/b9ee1a98b5d541dfb6ab76f440cd43e2/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:15:24.925378 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ce288-bcd5-4869-94ec-8cabacbb845e None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:24.925849 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ce288-bcd5-4869-94ec-8cabacbb845e None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:24.926135 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ce288-bcd5-4869-94ec-8cabacbb845e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:24.959469 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-a17ce288-bcd5-4869-94ec-8cabacbb845e None None] MFA Rules not processed for user `b9ee1a98b5d541dfb6ab76f440cd43e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:15:24.959469 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-a17ce288-bcd5-4869-94ec-8cabacbb845e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:25.024187 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1207/2366] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:15:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:15:25.036003 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c582fa2-3204-44a6-a575-4365987099ae admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:25.037878 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-0c582fa2-3204-44a6-a575-4365987099ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:25.102139 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c582fa2-3204-44a6-a575-4365987099ae admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:25.107360 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d6d5d2-3228-4643-aca6-87f68874a8a3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '635a410f471c43109438d0c39874070b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:25.108075 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d6d5d2-3228-4643-aca6-87f68874a8a3 service glance] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:25.108363 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d6d5d2-3228-4643-aca6-87f68874a8a3 service glance] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:25.108640 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d6d5d2-3228-4643-aca6-87f68874a8a3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:25.129088 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d6d5d2-3228-4643-aca6-87f68874a8a3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:25.131376 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d6d5d2-3228-4643-aca6-87f68874a8a3 service glance] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:25.143521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1160/2367] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:15:25 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:15:30.321462 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d090b3-38ad-4b78-9961-da214a81e530 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:30.321997 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d090b3-38ad-4b78-9961-da214a81e530 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:30.321997 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d090b3-38ad-4b78-9961-da214a81e530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:30.349610 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-e2d090b3-38ad-4b78-9961-da214a81e530 None None] MFA Rules not processed for user `8e302cf8d56c4db8bb8480ec234ac908`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:15:30.350901 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-e2d090b3-38ad-4b78-9961-da214a81e530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:30.385536 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:15:30.423318 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1208/2368] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:15:30 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:15:33.373267 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d6d5d2-3228-4643-aca6-87f68874a8a3 service glance] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:33.375255 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-03d6d5d2-3228-4643-aca6-87f68874a8a3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:33.550911 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d6d5d2-3228-4643-aca6-87f68874a8a3 service glance] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:33.555532 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c8d3c1-5a98-45e3-9f92-e681e27bcd88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:33.556438 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8d3c1-5a98-45e3-9f92-e681e27bcd88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:33.556721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8d3c1-5a98-45e3-9f92-e681e27bcd88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:33.556999 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c8d3c1-5a98-45e3-9f92-e681e27bcd88 admin admin] PATH_INFO: `/v3/users/49ee4818fef74e70bd1a352ab9c86e33` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:33.558203 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c8d3c1-5a98-45e3-9f92-e681e27bcd88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49ee4818fef74e70bd1a352ab9c86e33)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:33.560230 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c8d3c1-5a98-45e3-9f92-e681e27bcd88 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:33.725697 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-15c8d3c1-5a98-45e3-9f92-e681e27bcd88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49ee4818fef74e70bd1a352ab9c86e33'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:33.727027 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:15:33.742156 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-15c8d3c1-5a98-45e3-9f92-e681e27bcd88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49ee4818fef74e70bd1a352ab9c86e33'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:33.758062 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-15c8d3c1-5a98-45e3-9f92-e681e27bcd88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49ee4818fef74e70bd1a352ab9c86e33'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:33.758845 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-15c8d3c1-5a98-45e3-9f92-e681e27bcd88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49ee4818fef74e70bd1a352ab9c86e33'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:33.823108 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-15c8d3c1-5a98-45e3-9f92-e681e27bcd88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49ee4818fef74e70bd1a352ab9c86e33'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:33.829798 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1161/2369] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:15:33 2022] DELETE /identity/v3/users/49ee4818fef74e70bd1a352ab9c86e33 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:15:33.835111 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac07010-3d1f-4b48-9fb9-e3eff73d25ea None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:33.839350 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac07010-3d1f-4b48-9fb9-e3eff73d25ea None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:33.839350 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac07010-3d1f-4b48-9fb9-e3eff73d25ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:33.863374 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-3ac07010-3d1f-4b48-9fb9-e3eff73d25ea None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:15:33.864689 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-3ac07010-3d1f-4b48-9fb9-e3eff73d25ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:33.925603 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1209/2370] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:15:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:15:33.935596 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c8d3c1-5a98-45e3-9f92-e681e27bcd88 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:33.937028 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-15c8d3c1-5a98-45e3-9f92-e681e27bcd88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:34.059413 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c8d3c1-5a98-45e3-9f92-e681e27bcd88 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:34.064803 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7032eb1c-dd93-4e2f-ba83-94d67d516e7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:34.065451 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032eb1c-dd93-4e2f-ba83-94d67d516e7e service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:34.065748 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032eb1c-dd93-4e2f-ba83-94d67d516e7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:34.066024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7032eb1c-dd93-4e2f-ba83-94d67d516e7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:34.076660 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7032eb1c-dd93-4e2f-ba83-94d67d516e7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:34.078119 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7032eb1c-dd93-4e2f-ba83-94d67d516e7e service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:34.091809 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1162/2371] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:15:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:15:34.905409 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac07010-3d1f-4b48-9fb9-e3eff73d25ea None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:34.906877 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-3ac07010-3d1f-4b48-9fb9-e3eff73d25ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:34.921424 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac07010-3d1f-4b48-9fb9-e3eff73d25ea None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:34.927368 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586584c3-ba42-433d-88d9-5973770eaaa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:34.928278 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586584c3-ba42-433d-88d9-5973770eaaa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:34.928564 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586584c3-ba42-433d-88d9-5973770eaaa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:34.928869 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586584c3-ba42-433d-88d9-5973770eaaa5 admin admin] PATH_INFO: `/v3/projects/a98ed6fa4ade48e4a66cb733c2cb76d8` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:34.930119 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586584c3-ba42-433d-88d9-5973770eaaa5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a98ed6fa4ade48e4a66cb733c2cb76d8)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:34.932525 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586584c3-ba42-433d-88d9-5973770eaaa5 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:35.069912 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-586584c3-ba42-433d-88d9-5973770eaaa5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a98ed6fa4ade48e4a66cb733c2cb76d8', 'request_id': 'req-586584c3-ba42-433d-88d9-5973770eaaa5'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:35.077908 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-586584c3-ba42-433d-88d9-5973770eaaa5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a98ed6fa4ade48e4a66cb733c2cb76d8', 'request_id': 'req-586584c3-ba42-433d-88d9-5973770eaaa5'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:35.089489 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-586584c3-ba42-433d-88d9-5973770eaaa5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a98ed6fa4ade48e4a66cb733c2cb76d8', 'request_id': 'req-586584c3-ba42-433d-88d9-5973770eaaa5'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:35.181320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-586584c3-ba42-433d-88d9-5973770eaaa5 admin admin] The token cache is being invalidate because project a98ed6fa4ade48e4a66cb733c2cb76d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:15:35.186906 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-586584c3-ba42-433d-88d9-5973770eaaa5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:35.186906 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-586584c3-ba42-433d-88d9-5973770eaaa5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:35.186906 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1210/2372] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:15:34 2022] DELETE /identity/v3/projects/a98ed6fa4ade48e4a66cb733c2cb76d8 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:15:35.189244 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d671c7-b0e9-445c-bdc1-87ac5a50caeb None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:35.189765 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d671c7-b0e9-445c-bdc1-87ac5a50caeb None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:35.189859 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d671c7-b0e9-445c-bdc1-87ac5a50caeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:35.231123 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-71d671c7-b0e9-445c-bdc1-87ac5a50caeb None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:15:35.232386 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-71d671c7-b0e9-445c-bdc1-87ac5a50caeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:35.297556 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1163/2373] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:15:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:15:35.306807 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c386b04-e557-41de-ba18-a015135b904f None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:35.314739 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c386b04-e557-41de-ba18-a015135b904f None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:35.314739 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c386b04-e557-41de-ba18-a015135b904f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:35.330101 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-1c386b04-e557-41de-ba18-a015135b904f None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:15:35.331575 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-1c386b04-e557-41de-ba18-a015135b904f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:35.349585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1211/2374] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:15:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:15:35.350865 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d671c7-b0e9-445c-bdc1-87ac5a50caeb None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:35.363054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d671c7-b0e9-445c-bdc1-87ac5a50caeb None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:35.378960 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70495558-925d-474e-aca8-d7a8410c4672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:35.379656 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70495558-925d-474e-aca8-d7a8410c4672 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:35.380638 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70495558-925d-474e-aca8-d7a8410c4672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:35.380920 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70495558-925d-474e-aca8-d7a8410c4672 admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:35.381886 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70495558-925d-474e-aca8-d7a8410c4672 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:15:35.382168 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70495558-925d-474e-aca8-d7a8410c4672 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:35.383832 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70495558-925d-474e-aca8-d7a8410c4672 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:35.402309 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1164/2375] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:15:35.406943 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c386b04-e557-41de-ba18-a015135b904f None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:35.416757 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c386b04-e557-41de-ba18-a015135b904f None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:35.421687 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722269d4-b43e-465d-bf29-0b33601c3374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:35.423380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722269d4-b43e-465d-bf29-0b33601c3374 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:35.423380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722269d4-b43e-465d-bf29-0b33601c3374 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:35.423380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722269d4-b43e-465d-bf29-0b33601c3374 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:35.423668 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722269d4-b43e-465d-bf29-0b33601c3374 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:35.425607 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722269d4-b43e-465d-bf29-0b33601c3374 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:35.506026 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1212/2376] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:15:35 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:15:35.511269 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70495558-925d-474e-aca8-d7a8410c4672 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:35.518487 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:15:35.523446 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70495558-925d-474e-aca8-d7a8410c4672 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:35.584826 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056aac80-8cd5-4a19-9e04-ba06b9984979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:35.585658 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056aac80-8cd5-4a19-9e04-ba06b9984979 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:35.586529 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056aac80-8cd5-4a19-9e04-ba06b9984979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:35.586886 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056aac80-8cd5-4a19-9e04-ba06b9984979 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:35.589359 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056aac80-8cd5-4a19-9e04-ba06b9984979 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:35.591650 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056aac80-8cd5-4a19-9e04-ba06b9984979 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:35.679226 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1165/2377] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:15:35 2022] POST /identity/v3/users => generated 288 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:15:35.680096 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722269d4-b43e-465d-bf29-0b33601c3374 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:35.695417 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722269d4-b43e-465d-bf29-0b33601c3374 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:35.700107 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea6543-2b67-414d-9131-a966b8d3651a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:35.700808 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea6543-2b67-414d-9131-a966b8d3651a admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:35.701093 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea6543-2b67-414d-9131-a966b8d3651a admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:35.701366 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea6543-2b67-414d-9131-a966b8d3651a admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:35.702171 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea6543-2b67-414d-9131-a966b8d3651a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:35.703809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea6543-2b67-414d-9131-a966b8d3651a admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:35.715364 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1213/2378] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:15:35.719371 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056aac80-8cd5-4a19-9e04-ba06b9984979 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:35.729709 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056aac80-8cd5-4a19-9e04-ba06b9984979 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:35.734541 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c6b24c-b731-4db4-a5cc-f1b4dbbade88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:35.735214 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6b24c-b731-4db4-a5cc-f1b4dbbade88 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:35.735497 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6b24c-b731-4db4-a5cc-f1b4dbbade88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:35.735812 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6b24c-b731-4db4-a5cc-f1b4dbbade88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:35.737245 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c6b24c-b731-4db4-a5cc-f1b4dbbade88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:35.738939 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c6b24c-b731-4db4-a5cc-f1b4dbbade88 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:35.755614 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1166/2379] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:15:35.760442 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea6543-2b67-414d-9131-a966b8d3651a admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:35.767018 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:15:35.770975 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea6543-2b67-414d-9131-a966b8d3651a admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:35.775612 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a762ec-4500-4873-a6e7-2638b19e0b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:35.776787 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a762ec-4500-4873-a6e7-2638b19e0b52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:35.777067 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a762ec-4500-4873-a6e7-2638b19e0b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:35.783327 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a762ec-4500-4873-a6e7-2638b19e0b52 admin admin] PATH_INFO: `/v3/projects/c8641428a8434d449d13951e314e56ef/users/815f9aebb5394dad9ab4fe1a3f3af4ea/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:35.794108 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a762ec-4500-4873-a6e7-2638b19e0b52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8641428a8434d449d13951e314e56ef, user_id=815f9aebb5394dad9ab4fe1a3f3af4ea, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:35.796888 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a762ec-4500-4873-a6e7-2638b19e0b52 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:35.877141 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1214/2380] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:15:35 2022] PUT /identity/v3/projects/c8641428a8434d449d13951e314e56ef/users/815f9aebb5394dad9ab4fe1a3f3af4ea/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:15:35.963590 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba0e3a-81b6-4f2e-a418-ff55b0a70d5c None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:35.964516 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba0e3a-81b6-4f2e-a418-ff55b0a70d5c None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:35.964801 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ba0e3a-81b6-4f2e-a418-ff55b0a70d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:36.001889 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-35ba0e3a-81b6-4f2e-a418-ff55b0a70d5c None None] MFA Rules not processed for user `815f9aebb5394dad9ab4fe1a3f3af4ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:15:36.003101 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-35ba0e3a-81b6-4f2e-a418-ff55b0a70d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:36.070254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1167/2381] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:15:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:15:36.082288 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a762ec-4500-4873-a6e7-2638b19e0b52 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:36.084129 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d1a762ec-4500-4873-a6e7-2638b19e0b52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:36.144773 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a762ec-4500-4873-a6e7-2638b19e0b52 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:36.148440 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f4084-1239-420b-ba21-529f784906e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:36.149097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f4084-1239-420b-ba21-529f784906e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:36.149383 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f4084-1239-420b-ba21-529f784906e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:36.149663 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f4084-1239-420b-ba21-529f784906e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:36.158487 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896f4084-1239-420b-ba21-529f784906e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:36.159988 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896f4084-1239-420b-ba21-529f784906e1 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:36.170713 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1215/2382] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:15:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:15:37.856925 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136b646-36f0-4fc5-b8d7-00a21acb7b92 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:37.857389 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136b646-36f0-4fc5-b8d7-00a21acb7b92 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:37.857480 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136b646-36f0-4fc5-b8d7-00a21acb7b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:37.876156 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-3136b646-36f0-4fc5-b8d7-00a21acb7b92 None None] MFA Rules not processed for user `815f9aebb5394dad9ab4fe1a3f3af4ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:15:37.877345 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-3136b646-36f0-4fc5-b8d7-00a21acb7b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:37.885765 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1168/2383] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:15:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:15:45.684689 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f4084-1239-420b-ba21-529f784906e1 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:45.686184 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-896f4084-1239-420b-ba21-529f784906e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:45.754480 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f4084-1239-420b-ba21-529f784906e1 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:45.760124 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8e87f3-0239-4cce-8d49-0dff9732e4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:45.761051 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e87f3-0239-4cce-8d49-0dff9732e4d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:45.761342 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e87f3-0239-4cce-8d49-0dff9732e4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:45.761621 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e87f3-0239-4cce-8d49-0dff9732e4d3 admin admin] PATH_INFO: `/v3/users/b9ee1a98b5d541dfb6ab76f440cd43e2` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:45.762895 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8e87f3-0239-4cce-8d49-0dff9732e4d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9ee1a98b5d541dfb6ab76f440cd43e2)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:45.764905 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8e87f3-0239-4cce-8d49-0dff9732e4d3 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:45.992070 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-ce8e87f3-0239-4cce-8d49-0dff9732e4d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9ee1a98b5d541dfb6ab76f440cd43e2'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:45.999058 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-ce8e87f3-0239-4cce-8d49-0dff9732e4d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9ee1a98b5d541dfb6ab76f440cd43e2'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:45.999740 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-ce8e87f3-0239-4cce-8d49-0dff9732e4d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9ee1a98b5d541dfb6ab76f440cd43e2'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:46.013908 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-ce8e87f3-0239-4cce-8d49-0dff9732e4d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9ee1a98b5d541dfb6ab76f440cd43e2'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:46.079902 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-ce8e87f3-0239-4cce-8d49-0dff9732e4d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9ee1a98b5d541dfb6ab76f440cd43e2'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:46.086980 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1216/2384] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:15:45 2022] DELETE /identity/v3/users/b9ee1a98b5d541dfb6ab76f440cd43e2 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:15:46.091582 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ab316-a623-4fef-8ad4-d4d0cbbf2afe None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:46.092056 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ab316-a623-4fef-8ad4-d4d0cbbf2afe None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:46.092206 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084ab316-a623-4fef-8ad4-d4d0cbbf2afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:46.109886 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-084ab316-a623-4fef-8ad4-d4d0cbbf2afe None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:15:46.111088 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-084ab316-a623-4fef-8ad4-d4d0cbbf2afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:46.168035 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1169/2385] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:15:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:15:46.178248 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8e87f3-0239-4cce-8d49-0dff9732e4d3 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:46.179646 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-ce8e87f3-0239-4cce-8d49-0dff9732e4d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:46.184986 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:15:46.236474 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8e87f3-0239-4cce-8d49-0dff9732e4d3 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:46.241153 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e1182d-a0c3-42ce-a340-bea61a22274b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:46.241861 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e1182d-a0c3-42ce-a340-bea61a22274b service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:46.246889 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e1182d-a0c3-42ce-a340-bea61a22274b service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:46.246889 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e1182d-a0c3-42ce-a340-bea61a22274b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:46.251455 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e1182d-a0c3-42ce-a340-bea61a22274b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:46.252929 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e1182d-a0c3-42ce-a340-bea61a22274b service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:46.262428 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1217/2386] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:15:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:15:46.701544 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084ab316-a623-4fef-8ad4-d4d0cbbf2afe None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:46.703758 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-084ab316-a623-4fef-8ad4-d4d0cbbf2afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:46.716004 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084ab316-a623-4fef-8ad4-d4d0cbbf2afe None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:46.726496 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c4fb7-65df-4651-b0a0-d26b6ebde720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:46.727881 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c4fb7-65df-4651-b0a0-d26b6ebde720 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:46.728225 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c4fb7-65df-4651-b0a0-d26b6ebde720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:46.728680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711c4fb7-65df-4651-b0a0-d26b6ebde720 admin admin] PATH_INFO: `/v3/projects/7654ee6379f14e95aacd02ddb886378b` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:46.729967 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c4fb7-65df-4651-b0a0-d26b6ebde720 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7654ee6379f14e95aacd02ddb886378b)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:46.732021 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711c4fb7-65df-4651-b0a0-d26b6ebde720 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:46.733927 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:15:46.837736 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-711c4fb7-65df-4651-b0a0-d26b6ebde720 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7654ee6379f14e95aacd02ddb886378b', 'request_id': 'req-711c4fb7-65df-4651-b0a0-d26b6ebde720'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:46.844091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-711c4fb7-65df-4651-b0a0-d26b6ebde720 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7654ee6379f14e95aacd02ddb886378b', 'request_id': 'req-711c4fb7-65df-4651-b0a0-d26b6ebde720'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:46.913400 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-711c4fb7-65df-4651-b0a0-d26b6ebde720 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7654ee6379f14e95aacd02ddb886378b', 'request_id': 'req-711c4fb7-65df-4651-b0a0-d26b6ebde720'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:46.921869 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-711c4fb7-65df-4651-b0a0-d26b6ebde720 admin admin] The token cache is being invalidate because project 7654ee6379f14e95aacd02ddb886378b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:15:46.922157 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-711c4fb7-65df-4651-b0a0-d26b6ebde720 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:46.922802 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-711c4fb7-65df-4651-b0a0-d26b6ebde720 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:15:46.924745 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1170/2387] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:15:46 2022] DELETE /identity/v3/projects/7654ee6379f14e95aacd02ddb886378b => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:15:56.263588 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=25 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:15:56.530505 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e583955-ec62-46b7-9989-71485e4828e4 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:56.530801 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e583955-ec62-46b7-9989-71485e4828e4 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:56.531088 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e583955-ec62-46b7-9989-71485e4828e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:56.558560 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-4e583955-ec62-46b7-9989-71485e4828e4 None None] MFA Rules not processed for user `1905e6f8c3e24e3cbfefce2827baa370`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:15:56.559784 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-4e583955-ec62-46b7-9989-71485e4828e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:56.619856 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1218/2388] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 6 17:15:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:15:56.659546 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c4fb7-65df-4651-b0a0-d26b6ebde720 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:56.660989 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-711c4fb7-65df-4651-b0a0-d26b6ebde720 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:56.729047 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711c4fb7-65df-4651-b0a0-d26b6ebde720 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:56.734008 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f5b49-5d7d-46af-995d-292fcc1f159e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75acacd30e9a4cf2ad58b072b49e3f05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:56.734719 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f5b49-5d7d-46af-995d-292fcc1f159e service swift] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:56.734932 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f5b49-5d7d-46af-995d-292fcc1f159e service swift] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:56.735211 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1f5b49-5d7d-46af-995d-292fcc1f159e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:56.743592 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1f5b49-5d7d-46af-995d-292fcc1f159e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:56.745489 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1f5b49-5d7d-46af-995d-292fcc1f159e service swift] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:56.753427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1171/2389] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 6 17:15:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 17:15:57.584826 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4ad94-9802-47f1-a4e4-fc661abc50a4 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:57.585254 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4ad94-9802-47f1-a4e4-fc661abc50a4 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:57.585393 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b4ad94-9802-47f1-a4e4-fc661abc50a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:57.603365 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-d5b4ad94-9802-47f1-a4e4-fc661abc50a4 None None] MFA Rules not processed for user `1905e6f8c3e24e3cbfefce2827baa370`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:15:57.604588 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d5b4ad94-9802-47f1-a4e4-fc661abc50a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:57.612072 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1219/2390] 10.222.0.40 () {58 vars in 967 bytes} [Wed Jul 6 17:15:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:15:57.625238 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f5b49-5d7d-46af-995d-292fcc1f159e service swift] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:57.635617 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1f5b49-5d7d-46af-995d-292fcc1f159e service swift] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:57.640946 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c064c-61a6-4e20-bbc0-c128f461f108 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75acacd30e9a4cf2ad58b072b49e3f05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:57.641686 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c064c-61a6-4e20-bbc0-c128f461f108 service swift] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:57.641975 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c064c-61a6-4e20-bbc0-c128f461f108 service swift] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:57.642252 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c064c-61a6-4e20-bbc0-c128f461f108 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:57.650232 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0c064c-61a6-4e20-bbc0-c128f461f108 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:57.652145 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0c064c-61a6-4e20-bbc0-c128f461f108 service swift] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:57.660007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1172/2391] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 6 17:15:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 17:15:58.187323 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb1c6e-c92b-4b15-b651-8be3855745e1 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:58.187620 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb1c6e-c92b-4b15-b651-8be3855745e1 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:58.187945 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afb1c6e-c92b-4b15-b651-8be3855745e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:58.205730 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-4afb1c6e-c92b-4b15-b651-8be3855745e1 None None] MFA Rules not processed for user `1905e6f8c3e24e3cbfefce2827baa370`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:15:58.206909 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-4afb1c6e-c92b-4b15-b651-8be3855745e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:58.214998 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1220/2392] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 6 17:15:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:15:58.226716 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c064c-61a6-4e20-bbc0-c128f461f108 service swift] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:58.237109 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c064c-61a6-4e20-bbc0-c128f461f108 service swift] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:58.241997 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f63d5ff-1405-4157-875a-e0e81bbfa732 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75acacd30e9a4cf2ad58b072b49e3f05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:58.242660 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63d5ff-1405-4157-875a-e0e81bbfa732 service swift] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:58.242948 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63d5ff-1405-4157-875a-e0e81bbfa732 service swift] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:58.243227 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f63d5ff-1405-4157-875a-e0e81bbfa732 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:58.251175 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f63d5ff-1405-4157-875a-e0e81bbfa732 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:58.252805 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f63d5ff-1405-4157-875a-e0e81bbfa732 service swift] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:58.261299 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1173/2393] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 6 17:15:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 17:15:58.989560 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e864c5-f067-4672-9523-a159e6a0b5a2 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:58.990048 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e864c5-f067-4672-9523-a159e6a0b5a2 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:58.990147 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e864c5-f067-4672-9523-a159e6a0b5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:59.007649 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-18e864c5-f067-4672-9523-a159e6a0b5a2 None None] MFA Rules not processed for user `1905e6f8c3e24e3cbfefce2827baa370`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:15:59.009381 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-18e864c5-f067-4672-9523-a159e6a0b5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:15:59.016785 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1221/2394] 10.222.0.40 () {58 vars in 967 bytes} [Wed Jul 6 17:15:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:15:59.029806 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f63d5ff-1405-4157-875a-e0e81bbfa732 service swift] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:15:59.040840 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f63d5ff-1405-4157-875a-e0e81bbfa732 service swift] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:15:59.045920 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7057826-1666-49d0-999f-270e492ae634 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75acacd30e9a4cf2ad58b072b49e3f05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:15:59.046575 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7057826-1666-49d0-999f-270e492ae634 service swift] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:15:59.046866 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7057826-1666-49d0-999f-270e492ae634 service swift] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:15:59.047226 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7057826-1666-49d0-999f-270e492ae634 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:15:59.056462 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7057826-1666-49d0-999f-270e492ae634 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:15:59.058187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7057826-1666-49d0-999f-270e492ae634 service swift] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:15:59.069221 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1174/2395] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 6 17:15:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 17:16:01.929770 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e864c5-f067-4672-9523-a159e6a0b5a2 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:16:01.931879 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-18e864c5-f067-4672-9523-a159e6a0b5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:16:01.995670 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e864c5-f067-4672-9523-a159e6a0b5a2 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:16:02.000070 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d394f5-fd17-4134-9b00-3aa2dccbcf2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:16:02.000932 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d394f5-fd17-4134-9b00-3aa2dccbcf2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:16:02.001212 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d394f5-fd17-4134-9b00-3aa2dccbcf2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:16:02.001484 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d394f5-fd17-4134-9b00-3aa2dccbcf2e admin admin] PATH_INFO: `/v3/users/815f9aebb5394dad9ab4fe1a3f3af4ea` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:16:02.002684 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d394f5-fd17-4134-9b00-3aa2dccbcf2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=815f9aebb5394dad9ab4fe1a3f3af4ea)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:16:02.004630 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d394f5-fd17-4134-9b00-3aa2dccbcf2e admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:16:02.190866 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-79d394f5-fd17-4134-9b00-3aa2dccbcf2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '815f9aebb5394dad9ab4fe1a3f3af4ea'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:16:02.197728 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-79d394f5-fd17-4134-9b00-3aa2dccbcf2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '815f9aebb5394dad9ab4fe1a3f3af4ea'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:16:02.198395 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-79d394f5-fd17-4134-9b00-3aa2dccbcf2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '815f9aebb5394dad9ab4fe1a3f3af4ea'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:16:02.209164 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:16:02.210418 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-79d394f5-fd17-4134-9b00-3aa2dccbcf2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '815f9aebb5394dad9ab4fe1a3f3af4ea'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:16:02.285843 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-79d394f5-fd17-4134-9b00-3aa2dccbcf2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '815f9aebb5394dad9ab4fe1a3f3af4ea'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:16:02.292500 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1222/2396] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:16:01 2022] DELETE /identity/v3/users/815f9aebb5394dad9ab4fe1a3f3af4ea => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:16:02.300655 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa5aa0-eec0-4d01-ad57-8c7321f3269a None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:16:02.300655 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa5aa0-eec0-4d01-ad57-8c7321f3269a None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:16:02.300655 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fa5aa0-eec0-4d01-ad57-8c7321f3269a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:16:02.315607 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-42fa5aa0-eec0-4d01-ad57-8c7321f3269a None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:16:02.318107 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-42fa5aa0-eec0-4d01-ad57-8c7321f3269a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:16:02.354059 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:16:02.380252 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1175/2397] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:16:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:16:02.390571 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d394f5-fd17-4134-9b00-3aa2dccbcf2e admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:16:02.392065 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-79d394f5-fd17-4134-9b00-3aa2dccbcf2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:16:02.452351 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d394f5-fd17-4134-9b00-3aa2dccbcf2e admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:16:02.456157 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6ae2a2-bbc3-466e-a6d4-cf390267abc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:16:02.456806 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ae2a2-bbc3-466e-a6d4-cf390267abc3 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:16:02.457137 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ae2a2-bbc3-466e-a6d4-cf390267abc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:16:02.457408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6ae2a2-bbc3-466e-a6d4-cf390267abc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:16:02.467388 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ae2a2-bbc3-466e-a6d4-cf390267abc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:16:02.468848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6ae2a2-bbc3-466e-a6d4-cf390267abc3 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:16:02.478708 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1223/2398] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:16:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 17:16:02.731613 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa5aa0-eec0-4d01-ad57-8c7321f3269a None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:16:02.733069 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-42fa5aa0-eec0-4d01-ad57-8c7321f3269a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:16:02.749208 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fa5aa0-eec0-4d01-ad57-8c7321f3269a None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:16:02.754255 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a9639f-911c-469f-a2f6-69cd4b2fc076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:16:02.756954 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9639f-911c-469f-a2f6-69cd4b2fc076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:16:02.756954 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9639f-911c-469f-a2f6-69cd4b2fc076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:16:02.756954 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9639f-911c-469f-a2f6-69cd4b2fc076 admin admin] PATH_INFO: `/v3/projects/c8641428a8434d449d13951e314e56ef` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:16:02.756954 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a9639f-911c-469f-a2f6-69cd4b2fc076 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8641428a8434d449d13951e314e56ef)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:16:02.759406 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a9639f-911c-469f-a2f6-69cd4b2fc076 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:16:02.884451 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-a9a9639f-911c-469f-a2f6-69cd4b2fc076 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8641428a8434d449d13951e314e56ef', 'request_id': 'req-a9a9639f-911c-469f-a2f6-69cd4b2fc076'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:16:02.890703 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-a9a9639f-911c-469f-a2f6-69cd4b2fc076 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8641428a8434d449d13951e314e56ef', 'request_id': 'req-a9a9639f-911c-469f-a2f6-69cd4b2fc076'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:16:02.982442 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-a9a9639f-911c-469f-a2f6-69cd4b2fc076 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8641428a8434d449d13951e314e56ef', 'request_id': 'req-a9a9639f-911c-469f-a2f6-69cd4b2fc076'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:16:02.994022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-a9a9639f-911c-469f-a2f6-69cd4b2fc076 admin admin] The token cache is being invalidate because project c8641428a8434d449d13951e314e56ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:16:02.994309 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-a9a9639f-911c-469f-a2f6-69cd4b2fc076 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:16:02.994959 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-a9a9639f-911c-469f-a2f6-69cd4b2fc076 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:16:02.996533 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1176/2399] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:16:02 2022] DELETE /identity/v3/projects/c8641428a8434d449d13951e314e56ef => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:16:03.001839 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca2426-03a8-42b0-bf81-09e8ba6e856b None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:16:03.002326 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca2426-03a8-42b0-bf81-09e8ba6e856b None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:16:03.002437 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfca2426-03a8-42b0-bf81-09e8ba6e856b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:16:03.031617 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-bfca2426-03a8-42b0-bf81-09e8ba6e856b None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:16:03.044581 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-bfca2426-03a8-42b0-bf81-09e8ba6e856b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:16:03.093331 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1224/2400] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:16:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:16:03.103408 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e553433e-9790-4cfc-ac61-c9afe46c50b1 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:16:03.103710 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e553433e-9790-4cfc-ac61-c9afe46c50b1 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:16:03.104024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e553433e-9790-4cfc-ac61-c9afe46c50b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:16:03.122107 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-e553433e-9790-4cfc-ac61-c9afe46c50b1 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:16:03.123234 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e553433e-9790-4cfc-ac61-c9afe46c50b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:16:03.130971 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1177/2401] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:16:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:16:03.135088 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfca2426-03a8-42b0-bf81-09e8ba6e856b None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:16:03.144778 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfca2426-03a8-42b0-bf81-09e8ba6e856b None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:16:03.149779 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a13ef0-95fd-4780-8020-efeeda62aae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:16:03.150422 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a13ef0-95fd-4780-8020-efeeda62aae3 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:16:03.150705 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a13ef0-95fd-4780-8020-efeeda62aae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:16:03.150979 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a13ef0-95fd-4780-8020-efeeda62aae3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:16:03.151979 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a13ef0-95fd-4780-8020-efeeda62aae3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:16:03.152288 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a13ef0-95fd-4780-8020-efeeda62aae3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:16:03.153871 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a13ef0-95fd-4780-8020-efeeda62aae3 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:16:03.165895 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1225/2402] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:16:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:16:03.170099 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e553433e-9790-4cfc-ac61-c9afe46c50b1 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:16:03.179207 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e553433e-9790-4cfc-ac61-c9afe46c50b1 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:16:03.183848 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7655a-3f58-47e8-8e2d-97893ca0386b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:16:03.184493 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7655a-3f58-47e8-8e2d-97893ca0386b admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:16:03.184779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7655a-3f58-47e8-8e2d-97893ca0386b admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:16:03.185072 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7655a-3f58-47e8-8e2d-97893ca0386b admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:16:03.185745 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd7655a-3f58-47e8-8e2d-97893ca0386b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:16:03.187634 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd7655a-3f58-47e8-8e2d-97893ca0386b admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:16:03.268483 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1178/2403] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:16:03 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:16:03.277186 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a13ef0-95fd-4780-8020-efeeda62aae3 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:16:03.287173 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a13ef0-95fd-4780-8020-efeeda62aae3 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:16:03.341909 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b0760-fb16-476b-888d-2c6200011399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:16:03.342573 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b0760-fb16-476b-888d-2c6200011399 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:16:03.342870 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b0760-fb16-476b-888d-2c6200011399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:16:03.343130 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b0760-fb16-476b-888d-2c6200011399 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:16:03.343848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982b0760-fb16-476b-888d-2c6200011399 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:16:03.345810 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982b0760-fb16-476b-888d-2c6200011399 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:16:03.435796 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:16:03.441091 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1226/2404] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:16:03 2022] POST /identity/v3/users => generated 293 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:16:03.445242 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7655a-3f58-47e8-8e2d-97893ca0386b admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:16:03.456033 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7655a-3f58-47e8-8e2d-97893ca0386b admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:16:03.460581 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c383a-f3fe-4075-97ca-5fe5061af5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:16:03.461873 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c383a-f3fe-4075-97ca-5fe5061af5ac admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:16:03.462156 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c383a-f3fe-4075-97ca-5fe5061af5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:16:03.462426 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921c383a-f3fe-4075-97ca-5fe5061af5ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:16:03.463147 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921c383a-f3fe-4075-97ca-5fe5061af5ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:16:03.464774 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921c383a-f3fe-4075-97ca-5fe5061af5ac admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:16:03.474939 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1179/2405] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:16:03.478874 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b0760-fb16-476b-888d-2c6200011399 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:16:03.488569 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b0760-fb16-476b-888d-2c6200011399 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:16:03.493197 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7326d4-0956-4fca-b2c4-f3ff1686153c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:16:03.493843 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7326d4-0956-4fca-b2c4-f3ff1686153c admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:16:03.494125 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7326d4-0956-4fca-b2c4-f3ff1686153c admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:16:03.494402 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7326d4-0956-4fca-b2c4-f3ff1686153c admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:16:03.495123 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7326d4-0956-4fca-b2c4-f3ff1686153c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:16:03.496806 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7326d4-0956-4fca-b2c4-f3ff1686153c admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:16:03.507118 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1227/2406] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:16:03.511664 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c383a-f3fe-4075-97ca-5fe5061af5ac admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:16:03.521581 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921c383a-f3fe-4075-97ca-5fe5061af5ac admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:16:03.529116 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd28f3-f7e3-4d94-913f-e3c78142ba53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:16:03.529974 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd28f3-f7e3-4d94-913f-e3c78142ba53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:16:03.530289 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd28f3-f7e3-4d94-913f-e3c78142ba53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:16:03.530569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd28f3-f7e3-4d94-913f-e3c78142ba53 admin admin] PATH_INFO: `/v3/projects/966f22e2bdf44625a350f3647b281d70/users/43974c97019b47908a00fc7261fc22a2/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:16:03.545042 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bd28f3-f7e3-4d94-913f-e3c78142ba53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=966f22e2bdf44625a350f3647b281d70, user_id=43974c97019b47908a00fc7261fc22a2, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:16:03.547868 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bd28f3-f7e3-4d94-913f-e3c78142ba53 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:16:03.593926 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1180/2407] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:16:03 2022] PUT /identity/v3/projects/966f22e2bdf44625a350f3647b281d70/users/43974c97019b47908a00fc7261fc22a2/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:16:03.599012 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5468437-c524-48a8-9f17-5d4efb736fa2 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:16:03.599546 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5468437-c524-48a8-9f17-5d4efb736fa2 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:16:03.599867 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5468437-c524-48a8-9f17-5d4efb736fa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:16:03.617160 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-f5468437-c524-48a8-9f17-5d4efb736fa2 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:16:03.618367 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f5468437-c524-48a8-9f17-5d4efb736fa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:16:03.675414 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1228/2408] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:16:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:16:03.684851 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd28f3-f7e3-4d94-913f-e3c78142ba53 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:16:03.686469 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-65bd28f3-f7e3-4d94-913f-e3c78142ba53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:16:03.748270 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd28f3-f7e3-4d94-913f-e3c78142ba53 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:16:03.751915 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e17f0d-23d5-4289-bfc7-a4f89fa1deaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:16:03.752568 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e17f0d-23d5-4289-bfc7-a4f89fa1deaf service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:16:03.752854 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e17f0d-23d5-4289-bfc7-a4f89fa1deaf service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:16:03.753137 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e17f0d-23d5-4289-bfc7-a4f89fa1deaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:16:03.762626 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e17f0d-23d5-4289-bfc7-a4f89fa1deaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:16:03.764102 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e17f0d-23d5-4289-bfc7-a4f89fa1deaf service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:16:03.773371 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1181/2409] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:16:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:16:08.906612 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c2b21-5680-4235-bab5-508b07b653fa None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:16:08.907173 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c2b21-5680-4235-bab5-508b07b653fa None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:16:08.907385 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c2b21-5680-4235-bab5-508b07b653fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:16:08.943264 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-7a3c2b21-5680-4235-bab5-508b07b653fa None None] MFA Rules not processed for user `43974c97019b47908a00fc7261fc22a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:16:08.944724 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-7a3c2b21-5680-4235-bab5-508b07b653fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:16:09.032895 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1229/2410] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:16:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:16:09.037779 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e17f0d-23d5-4289-bfc7-a4f89fa1deaf service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:16:09.044663 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:16:09.048884 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e17f0d-23d5-4289-bfc7-a4f89fa1deaf service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:16:09.054007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d280572-e0cf-4222-9985-8979e26a1151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:16:09.056189 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d280572-e0cf-4222-9985-8979e26a1151 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:16:09.056189 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d280572-e0cf-4222-9985-8979e26a1151 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:16:09.056189 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d280572-e0cf-4222-9985-8979e26a1151 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:16:09.056581 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d280572-e0cf-4222-9985-8979e26a1151 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:16:09.058496 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d280572-e0cf-4222-9985-8979e26a1151 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:16:09.165771 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1182/2411] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:16:09 2022] POST /identity/v3/projects => generated 383 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:16:09.168739 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3c2b21-5680-4235-bab5-508b07b653fa None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:16:09.180858 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3c2b21-5680-4235-bab5-508b07b653fa None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:16:09.249138 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924284b9-68a3-4b84-9496-64bd85d77b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:16:09.249738 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924284b9-68a3-4b84-9496-64bd85d77b64 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:16:09.250028 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924284b9-68a3-4b84-9496-64bd85d77b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:16:09.250308 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924284b9-68a3-4b84-9496-64bd85d77b64 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:16:09.251010 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924284b9-68a3-4b84-9496-64bd85d77b64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:16:09.253003 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924284b9-68a3-4b84-9496-64bd85d77b64 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:16:09.341479 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1230/2412] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:16:09 2022] POST /identity/v3/users => generated 298 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:16:09.342621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d280572-e0cf-4222-9985-8979e26a1151 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:16:09.353855 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d280572-e0cf-4222-9985-8979e26a1151 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:16:09.358525 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f4314-f518-418a-8b23-8ebb7b0538cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:16:09.359209 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f4314-f518-418a-8b23-8ebb7b0538cb admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:16:09.359495 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f4314-f518-418a-8b23-8ebb7b0538cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:16:09.359838 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f4314-f518-418a-8b23-8ebb7b0538cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:16:09.360569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055f4314-f518-418a-8b23-8ebb7b0538cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:16:09.362184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055f4314-f518-418a-8b23-8ebb7b0538cb admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:16:09.377008 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1183/2413] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:16:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:16:09.382872 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924284b9-68a3-4b84-9496-64bd85d77b64 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:16:09.394116 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924284b9-68a3-4b84-9496-64bd85d77b64 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:16:09.399311 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3beaca-02c6-4417-b3c6-e8b597f96819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:16:09.400261 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3beaca-02c6-4417-b3c6-e8b597f96819 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:16:09.400546 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3beaca-02c6-4417-b3c6-e8b597f96819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:16:09.400828 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3beaca-02c6-4417-b3c6-e8b597f96819 admin admin] PATH_INFO: `/v3/projects/71dd7e58775b460ebacf06d40f46fcc5/users/8206715fa699482c9688bf24028b4001/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:16:09.404643 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:16:09.418042 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3beaca-02c6-4417-b3c6-e8b597f96819 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71dd7e58775b460ebacf06d40f46fcc5, user_id=8206715fa699482c9688bf24028b4001, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:16:09.420832 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3beaca-02c6-4417-b3c6-e8b597f96819 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:16:09.481461 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1231/2414] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:16:09 2022] PUT /identity/v3/projects/71dd7e58775b460ebacf06d40f46fcc5/users/8206715fa699482c9688bf24028b4001/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:16:09.485668 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f4314-f518-418a-8b23-8ebb7b0538cb admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:16:09.496645 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f4314-f518-418a-8b23-8ebb7b0538cb admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:16:09.557781 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaae9ba-022a-4340-915d-554eb31ba4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:16:09.559013 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaae9ba-022a-4340-915d-554eb31ba4bf admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:16:09.559013 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaae9ba-022a-4340-915d-554eb31ba4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:16:09.559013 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaae9ba-022a-4340-915d-554eb31ba4bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:16:09.559756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaae9ba-022a-4340-915d-554eb31ba4bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:16:09.561403 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaae9ba-022a-4340-915d-554eb31ba4bf admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:16:09.572356 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1184/2415] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:16:09 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:16:09.576837 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3beaca-02c6-4417-b3c6-e8b597f96819 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:16:09.588043 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3beaca-02c6-4417-b3c6-e8b597f96819 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:16:09.592783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c94808f-118d-4974-8822-97a0c49f8d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:16:09.593676 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94808f-118d-4974-8822-97a0c49f8d99 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:16:09.593959 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94808f-118d-4974-8822-97a0c49f8d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:16:09.594242 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c94808f-118d-4974-8822-97a0c49f8d99 admin admin] PATH_INFO: `/v3/projects/71dd7e58775b460ebacf06d40f46fcc5/users/8206715fa699482c9688bf24028b4001/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:16:09.596387 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c94808f-118d-4974-8822-97a0c49f8d99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71dd7e58775b460ebacf06d40f46fcc5, user_id=8206715fa699482c9688bf24028b4001, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:16:09.599195 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c94808f-118d-4974-8822-97a0c49f8d99 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:16:09.610877 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1232/2416] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:16:09 2022] PUT /identity/v3/projects/71dd7e58775b460ebacf06d40f46fcc5/users/8206715fa699482c9688bf24028b4001/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:16:15.027806 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba1468f-79ed-4384-b12c-65cec2a21f5d None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:16:15.028268 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba1468f-79ed-4384-b12c-65cec2a21f5d None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:16:15.028382 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba1468f-79ed-4384-b12c-65cec2a21f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:16:15.058277 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-3ba1468f-79ed-4384-b12c-65cec2a21f5d None None] MFA Rules not processed for user `8206715fa699482c9688bf24028b4001`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:16:15.059584 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-3ba1468f-79ed-4384-b12c-65cec2a21f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:16:15.146185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1185/2417] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:16:15 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:16:15.159555 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c94808f-118d-4974-8822-97a0c49f8d99 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:16:15.162084 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-9c94808f-118d-4974-8822-97a0c49f8d99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:16:15.221718 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c94808f-118d-4974-8822-97a0c49f8d99 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:16:15.225281 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207b4a7-9148-4bea-8b53-b00f3faf3fec service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '635a410f471c43109438d0c39874070b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:16:15.225925 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207b4a7-9148-4bea-8b53-b00f3faf3fec service glance] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:16:15.226216 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207b4a7-9148-4bea-8b53-b00f3faf3fec service glance] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:16:15.226524 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207b4a7-9148-4bea-8b53-b00f3faf3fec service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:16:15.286675 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1207b4a7-9148-4bea-8b53-b00f3faf3fec service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:16:15.288187 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1207b4a7-9148-4bea-8b53-b00f3faf3fec service glance] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:16:15.311824 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1233/2418] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:16:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 17:16:15.579799 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7474df-4441-4914-b56b-b9285f15d77e None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:16:15.580155 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7474df-4441-4914-b56b-b9285f15d77e None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:16:15.580443 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7474df-4441-4914-b56b-b9285f15d77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:16:15.598417 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-7a7474df-4441-4914-b56b-b9285f15d77e None None] MFA Rules not processed for user `1905e6f8c3e24e3cbfefce2827baa370`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:16:15.599618 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-7a7474df-4441-4914-b56b-b9285f15d77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:16:15.649395 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:16:15.660608 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1186/2419] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 6 17:16:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:16:15.673515 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207b4a7-9148-4bea-8b53-b00f3faf3fec service glance] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:16:15.678480 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-1207b4a7-9148-4bea-8b53-b00f3faf3fec service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:16:15.750020 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207b4a7-9148-4bea-8b53-b00f3faf3fec service glance] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:16:15.755375 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ef65db-10fd-4db0-ab70-376aa2f84e46 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75acacd30e9a4cf2ad58b072b49e3f05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:16:15.756082 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef65db-10fd-4db0-ab70-376aa2f84e46 service swift] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:16:15.756367 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef65db-10fd-4db0-ab70-376aa2f84e46 service swift] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:16:15.756678 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ef65db-10fd-4db0-ab70-376aa2f84e46 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:16:15.765671 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ef65db-10fd-4db0-ab70-376aa2f84e46 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:16:15.767279 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ef65db-10fd-4db0-ab70-376aa2f84e46 service swift] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:16:15.775961 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1234/2420] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 6 17:16:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 17:16:19.408881 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7474df-4441-4914-b56b-b9285f15d77e None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:16:19.410369 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-7a7474df-4441-4914-b56b-b9285f15d77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:16:19.522990 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7474df-4441-4914-b56b-b9285f15d77e None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:16:19.532171 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dea6a8-f36c-4c4a-aff3-d0c3aa722889 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76ca877bee9041d28ad41e45ef3dd18c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:16:19.533102 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dea6a8-f36c-4c4a-aff3-d0c3aa722889 service placement] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:16:19.533549 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dea6a8-f36c-4c4a-aff3-d0c3aa722889 service placement] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:16:19.534003 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dea6a8-f36c-4c4a-aff3-d0c3aa722889 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:16:19.536554 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-a3dea6a8-f36c-4c4a-aff3-d0c3aa722889 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:16:19.559516 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dea6a8-f36c-4c4a-aff3-d0c3aa722889 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:16:19.561187 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dea6a8-f36c-4c4a-aff3-d0c3aa722889 service placement] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:16:19.583857 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1187/2421] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:16:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:16:25.772752 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=91,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:16:26.484787 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08ced2-6103-4eae-9933-c73f490582f3 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:16:26.485465 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08ced2-6103-4eae-9933-c73f490582f3 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:16:26.485465 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08ced2-6103-4eae-9933-c73f490582f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:16:26.503888 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-9a08ced2-6103-4eae-9933-c73f490582f3 None None] MFA Rules not processed for user `43974c97019b47908a00fc7261fc22a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:16:26.505117 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-9a08ced2-6103-4eae-9933-c73f490582f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:16:26.512670 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1235/2422] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:16:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:16:27.951509 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76ef61-9421-4989-87b6-815b6c99fb5d None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:16:27.952034 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76ef61-9421-4989-87b6-815b6c99fb5d None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:16:27.952127 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76ef61-9421-4989-87b6-815b6c99fb5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:16:28.000621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-0e76ef61-9421-4989-87b6-815b6c99fb5d None None] MFA Rules not processed for user `1905e6f8c3e24e3cbfefce2827baa370`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:16:28.001640 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-0e76ef61-9421-4989-87b6-815b6c99fb5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:16:28.019848 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1188/2423] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 6 17:16:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:16:28.032148 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08ced2-6103-4eae-9933-c73f490582f3 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:16:28.054834 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08ced2-6103-4eae-9933-c73f490582f3 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:16:28.060668 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a054f6-523d-479c-8a81-c37db3946688 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75acacd30e9a4cf2ad58b072b49e3f05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:16:28.061317 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a054f6-523d-479c-8a81-c37db3946688 service swift] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:16:28.061610 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a054f6-523d-479c-8a81-c37db3946688 service swift] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:16:28.061893 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a054f6-523d-479c-8a81-c37db3946688 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:16:28.070795 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a054f6-523d-479c-8a81-c37db3946688 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:16:28.072554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a054f6-523d-479c-8a81-c37db3946688 service swift] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:16:28.081058 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1236/2424] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 6 17:16:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 17:16:30.026775 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8485d4b-cf16-4177-8c6e-28ff9aa58297 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:16:30.028007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8485d4b-cf16-4177-8c6e-28ff9aa58297 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:16:30.028683 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8485d4b-cf16-4177-8c6e-28ff9aa58297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:16:30.060694 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-b8485d4b-cf16-4177-8c6e-28ff9aa58297 None None] MFA Rules not processed for user `1905e6f8c3e24e3cbfefce2827baa370`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:16:30.062120 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-b8485d4b-cf16-4177-8c6e-28ff9aa58297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:16:30.081003 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1189/2425] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 6 17:16:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:16:30.100028 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a054f6-523d-479c-8a81-c37db3946688 service swift] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:16:30.118874 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a054f6-523d-479c-8a81-c37db3946688 service swift] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:16:30.132847 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bb4416-b40c-4567-b472-9f57fc082e22 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75acacd30e9a4cf2ad58b072b49e3f05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:16:30.133867 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb4416-b40c-4567-b472-9f57fc082e22 service swift] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:16:30.134281 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb4416-b40c-4567-b472-9f57fc082e22 service swift] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:16:30.134746 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bb4416-b40c-4567-b472-9f57fc082e22 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:16:30.169799 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb4416-b40c-4567-b472-9f57fc082e22 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:16:30.171633 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bb4416-b40c-4567-b472-9f57fc082e22 service swift] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:16:30.184711 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1237/2426] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 6 17:16:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 17:16:40.046533 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=64 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:16:40.180042 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=19 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:16:48.659478 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2163588-6525-4375-9423-cf4291de032a None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:16:48.660250 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2163588-6525-4375-9423-cf4291de032a None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:16:48.660250 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2163588-6525-4375-9423-cf4291de032a None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:16:48.661842 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1190/2427] 10.222.0.40 () {54 vars in 921 bytes} [Wed Jul 6 17:16:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:16:48.668809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f318365-cd36-4d7f-846e-3004952db95e None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:16:48.669334 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f318365-cd36-4d7f-846e-3004952db95e None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:16:48.669437 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f318365-cd36-4d7f-846e-3004952db95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:16:48.726833 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-5f318365-cd36-4d7f-846e-3004952db95e None None] MFA Rules not processed for user `af26ef96dc9d47dc91549b736939268e`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:16:48.728258 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-5f318365-cd36-4d7f-846e-3004952db95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:16:48.813172 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1238/2428] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Jul 6 17:16:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:16:48.825798 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2163588-6525-4375-9423-cf4291de032a None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:16:48.827698 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e2163588-6525-4375-9423-cf4291de032a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:16:48.863543 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2163588-6525-4375-9423-cf4291de032a None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:16:48.867579 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0f0f38-d5f7-4801-9bb3-1d74a95c7a9b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '635a410f471c43109438d0c39874070b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:16:48.868279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f0f38-d5f7-4801-9bb3-1d74a95c7a9b service glance] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:16:48.868572 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f0f38-d5f7-4801-9bb3-1d74a95c7a9b service glance] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:16:48.868852 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0f0f38-d5f7-4801-9bb3-1d74a95c7a9b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:16:48.880399 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0f0f38-d5f7-4801-9bb3-1d74a95c7a9b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:16:48.882424 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0f0f38-d5f7-4801-9bb3-1d74a95c7a9b service glance] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:16:48.896386 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1191/2429] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:16:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:16:49.058858 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e8c466-9904-46a3-95bc-2f81802a3d8a None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:16:49.059175 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e8c466-9904-46a3-95bc-2f81802a3d8a None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:16:49.059458 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e8c466-9904-46a3-95bc-2f81802a3d8a None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:16:49.060875 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1239/2430] 10.222.0.40 () {54 vars in 921 bytes} [Wed Jul 6 17:16:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:16:49.065366 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b1660-9c2e-4c54-8413-572fdcb8483e None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:16:49.065676 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b1660-9c2e-4c54-8413-572fdcb8483e None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:16:49.065958 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b1660-9c2e-4c54-8413-572fdcb8483e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:16:49.090898 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-b51b1660-9c2e-4c54-8413-572fdcb8483e None None] MFA Rules not processed for user `5f479458a68b4daf9f3a7ae53d6c3763`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:16:49.092138 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-b51b1660-9c2e-4c54-8413-572fdcb8483e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:16:49.141838 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1192/2431] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Jul 6 17:16:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:16:49.154394 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e8c466-9904-46a3-95bc-2f81802a3d8a None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:16:49.156198 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-35e8c466-9904-46a3-95bc-2f81802a3d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:16:49.173076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e8c466-9904-46a3-95bc-2f81802a3d8a None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:16:49.176623 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5196b2b6-329b-48db-a88b-2905dedb9f87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:16:49.177264 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5196b2b6-329b-48db-a88b-2905dedb9f87 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:16:49.177552 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5196b2b6-329b-48db-a88b-2905dedb9f87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:16:49.177833 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5196b2b6-329b-48db-a88b-2905dedb9f87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:16:49.184873 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5196b2b6-329b-48db-a88b-2905dedb9f87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:16:49.186249 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5196b2b6-329b-48db-a88b-2905dedb9f87 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:16:49.193988 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1240/2432] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:16:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:16:49.279088 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81dfe8b-fdaa-4707-a0bc-203b02944770 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:16:49.279382 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81dfe8b-fdaa-4707-a0bc-203b02944770 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:16:49.279661 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81dfe8b-fdaa-4707-a0bc-203b02944770 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:16:49.281089 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1193/2433] 10.222.0.40 () {54 vars in 921 bytes} [Wed Jul 6 17:16:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:16:49.286253 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e2b96-0678-4aa2-a365-82fed1431d6b None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:16:49.286540 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e2b96-0678-4aa2-a365-82fed1431d6b None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:16:49.286815 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4e2b96-0678-4aa2-a365-82fed1431d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:16:49.305437 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-aa4e2b96-0678-4aa2-a365-82fed1431d6b None None] MFA Rules not processed for user `af26ef96dc9d47dc91549b736939268e`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:16:49.306649 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-aa4e2b96-0678-4aa2-a365-82fed1431d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:16:49.316034 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1241/2434] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Jul 6 17:16:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:16:49.326677 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81dfe8b-fdaa-4707-a0bc-203b02944770 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:16:49.338371 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81dfe8b-fdaa-4707-a0bc-203b02944770 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:16:49.342373 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13caafb-34ee-475c-9b5d-4672527517a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:16:49.343122 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13caafb-34ee-475c-9b5d-4672527517a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:16:49.344550 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13caafb-34ee-475c-9b5d-4672527517a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:16:49.344550 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13caafb-34ee-475c-9b5d-4672527517a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:16:49.355119 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13caafb-34ee-475c-9b5d-4672527517a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:16:49.356649 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13caafb-34ee-475c-9b5d-4672527517a4 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:16:49.368789 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1194/2435] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:16:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:16:53.352413 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2828ded7-f998-4800-8cda-0d15b0bd4e46 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:16:53.352902 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2828ded7-f998-4800-8cda-0d15b0bd4e46 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:16:53.353011 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2828ded7-f998-4800-8cda-0d15b0bd4e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:16:53.371877 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-2828ded7-f998-4800-8cda-0d15b0bd4e46 None None] MFA Rules not processed for user `1905e6f8c3e24e3cbfefce2827baa370`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:16:53.377676 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-2828ded7-f998-4800-8cda-0d15b0bd4e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:16:53.381297 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1242/2436] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 6 17:16:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:16:53.393791 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13caafb-34ee-475c-9b5d-4672527517a4 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:16:53.405446 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13caafb-34ee-475c-9b5d-4672527517a4 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:16:53.411242 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8086abab-a309-4264-bce6-24e0d31089f8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75acacd30e9a4cf2ad58b072b49e3f05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:16:53.413823 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086abab-a309-4264-bce6-24e0d31089f8 service swift] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:16:53.413823 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086abab-a309-4264-bce6-24e0d31089f8 service swift] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:16:53.413823 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8086abab-a309-4264-bce6-24e0d31089f8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:16:53.422007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8086abab-a309-4264-bce6-24e0d31089f8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:16:53.423603 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8086abab-a309-4264-bce6-24e0d31089f8 service swift] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:16:53.433375 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1195/2437] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 6 17:16:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 17:17:03.365977 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=58 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:17:03.428770 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=50 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:17:11.199278 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2828ded7-f998-4800-8cda-0d15b0bd4e46 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:17:11.208738 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2828ded7-f998-4800-8cda-0d15b0bd4e46 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:17:11.212879 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d41029-0e83-42e1-9bc0-66821a358582 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:17:11.213527 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d41029-0e83-42e1-9bc0-66821a358582 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:17:11.213813 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d41029-0e83-42e1-9bc0-66821a358582 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:17:11.214092 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d41029-0e83-42e1-9bc0-66821a358582 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:17:11.223310 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d41029-0e83-42e1-9bc0-66821a358582 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:17:11.224759 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d41029-0e83-42e1-9bc0-66821a358582 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:17:11.236381 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1243/2438] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:17:11 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:17:13.820729 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63279b-476f-49cb-8fc8-98438e6a1d3b None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:17:13.826212 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63279b-476f-49cb-8fc8-98438e6a1d3b None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:17:13.826212 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c63279b-476f-49cb-8fc8-98438e6a1d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:17:13.841232 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-6c63279b-476f-49cb-8fc8-98438e6a1d3b None None] MFA Rules not processed for user `1905e6f8c3e24e3cbfefce2827baa370`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:17:13.842840 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-6c63279b-476f-49cb-8fc8-98438e6a1d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:17:13.852093 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1196/2439] 10.222.0.40 () {58 vars in 967 bytes} [Wed Jul 6 17:17:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:17:13.880788 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d41029-0e83-42e1-9bc0-66821a358582 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:17:13.895623 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d41029-0e83-42e1-9bc0-66821a358582 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:17:13.900999 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b1aa07-4f22-43c6-856e-c419f63e1744 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75acacd30e9a4cf2ad58b072b49e3f05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:17:13.901880 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1aa07-4f22-43c6-856e-c419f63e1744 service swift] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:17:13.902173 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1aa07-4f22-43c6-856e-c419f63e1744 service swift] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:17:13.902453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b1aa07-4f22-43c6-856e-c419f63e1744 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:17:13.913045 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b1aa07-4f22-43c6-856e-c419f63e1744 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:17:13.915865 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b1aa07-4f22-43c6-856e-c419f63e1744 service swift] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:17:13.926260 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1244/2440] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 6 17:17:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 17:17:23.836078 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:17:23.921880 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=12 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:17:26.300898 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee6d3d-0db1-4e07-b3bc-393bc8ff73bc None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:17:26.301443 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee6d3d-0db1-4e07-b3bc-393bc8ff73bc None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:17:26.301443 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ee6d3d-0db1-4e07-b3bc-393bc8ff73bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:17:26.320847 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-42ee6d3d-0db1-4e07-b3bc-393bc8ff73bc None None] MFA Rules not processed for user `8e302cf8d56c4db8bb8480ec234ac908`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:17:26.322032 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-42ee6d3d-0db1-4e07-b3bc-393bc8ff73bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:17:26.374854 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1197/2441] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:17:26 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:17:36.368102 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=25 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:17:51.782042 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a219d6-dd7f-45b8-b515-a91bf41e7f6d None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:17:51.782616 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a219d6-dd7f-45b8-b515-a91bf41e7f6d None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:17:51.782616 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a219d6-dd7f-45b8-b515-a91bf41e7f6d None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:17:51.784140 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1245/2442] 10.222.0.40 () {54 vars in 921 bytes} [Wed Jul 6 17:17:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:17:51.792958 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda408e-52b3-48ae-b2ce-dfc64d9e6475 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:17:51.793510 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda408e-52b3-48ae-b2ce-dfc64d9e6475 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:17:51.793879 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda408e-52b3-48ae-b2ce-dfc64d9e6475 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:17:51.821865 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-ceda408e-52b3-48ae-b2ce-dfc64d9e6475 None None] MFA Rules not processed for user `5f479458a68b4daf9f3a7ae53d6c3763`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:17:51.823247 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-ceda408e-52b3-48ae-b2ce-dfc64d9e6475 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:17:51.839432 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1198/2443] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Jul 6 17:17:51 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:17:51.858016 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a219d6-dd7f-45b8-b515-a91bf41e7f6d None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:17:51.879297 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a219d6-dd7f-45b8-b515-a91bf41e7f6d None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:17:51.885608 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec5d8f3-7682-4daa-a659-9a15f4649346 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:17:51.889245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec5d8f3-7682-4daa-a659-9a15f4649346 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:17:51.889245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec5d8f3-7682-4daa-a659-9a15f4649346 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:17:51.889245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec5d8f3-7682-4daa-a659-9a15f4649346 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:17:51.898571 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec5d8f3-7682-4daa-a659-9a15f4649346 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:17:51.901061 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec5d8f3-7682-4daa-a659-9a15f4649346 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:17:51.918186 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1246/2444] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:17:51 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:17:52.021847 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e81e14-e168-4b0f-aa07-ad1e5043fb55 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:17:52.022144 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e81e14-e168-4b0f-aa07-ad1e5043fb55 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:17:52.022423 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e81e14-e168-4b0f-aa07-ad1e5043fb55 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:17:52.023825 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1199/2445] 10.222.0.40 () {54 vars in 921 bytes} [Wed Jul 6 17:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:17:52.030887 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e97eb4-2f50-4937-add4-c0fa5f25b0fa None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:17:52.033496 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e97eb4-2f50-4937-add4-c0fa5f25b0fa None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:17:52.033496 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e97eb4-2f50-4937-add4-c0fa5f25b0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:17:52.050409 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-24e97eb4-2f50-4937-add4-c0fa5f25b0fa None None] MFA Rules not processed for user `af26ef96dc9d47dc91549b736939268e`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:17:52.051708 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-24e97eb4-2f50-4937-add4-c0fa5f25b0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:17:52.061452 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1247/2446] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Jul 6 17:17:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:17:52.072689 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e81e14-e168-4b0f-aa07-ad1e5043fb55 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:17:52.083565 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e81e14-e168-4b0f-aa07-ad1e5043fb55 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:17:52.087046 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed5a5ea-9acc-418b-9433-eb8391089524 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:17:52.087706 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5a5ea-9acc-418b-9433-eb8391089524 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:17:52.088601 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5a5ea-9acc-418b-9433-eb8391089524 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:17:52.094616 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed5a5ea-9acc-418b-9433-eb8391089524 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:17:52.099408 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed5a5ea-9acc-418b-9433-eb8391089524 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:17:52.100889 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed5a5ea-9acc-418b-9433-eb8391089524 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:17:52.111820 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1200/2447] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:17:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:17:57.195477 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a75ef-2d0a-4eea-8442-e453a6659b01 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:17:57.199782 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a75ef-2d0a-4eea-8442-e453a6659b01 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:17:57.199782 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a75ef-2d0a-4eea-8442-e453a6659b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:17:57.214571 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-cb2a75ef-2d0a-4eea-8442-e453a6659b01 None None] MFA Rules not processed for user `7922db6ad0ce41868a68f5a834a3b8f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:17:57.215834 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-cb2a75ef-2d0a-4eea-8442-e453a6659b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:17:57.269646 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1248/2448] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:17:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:18:02.106954 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=9 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:18:06.618388 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64923ef5-3918-400e-beb4-f903828c334f None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:06.618981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64923ef5-3918-400e-beb4-f903828c334f None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:06.618981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64923ef5-3918-400e-beb4-f903828c334f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:06.636739 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-64923ef5-3918-400e-beb4-f903828c334f None None] MFA Rules not processed for user `8b5fdc57573c421195439a0c8e95d7d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:18:06.637904 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-64923ef5-3918-400e-beb4-f903828c334f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:18:06.692101 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1201/2449] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:18:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:18:07.262968 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=34 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:18:09.630810 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a75ef-2d0a-4eea-8442-e453a6659b01 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:18:09.640618 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a75ef-2d0a-4eea-8442-e453a6659b01 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:18:09.644293 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da1b1a1-48de-461b-a0da-e63318ada988 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:18:09.644976 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da1b1a1-48de-461b-a0da-e63318ada988 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:09.645313 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da1b1a1-48de-461b-a0da-e63318ada988 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:09.645596 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da1b1a1-48de-461b-a0da-e63318ada988 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:09.647635 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-3da1b1a1-48de-461b-a0da-e63318ada988 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:18:09.663100 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da1b1a1-48de-461b-a0da-e63318ada988 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:18:09.664670 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da1b1a1-48de-461b-a0da-e63318ada988 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:18:09.676951 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1249/2450] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:18:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:18:11.944731 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64923ef5-3918-400e-beb4-f903828c334f None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:18:11.959775 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64923ef5-3918-400e-beb4-f903828c334f None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:18:11.965735 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e946e64c-dd62-4836-8cad-99bd99130cc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:18:11.966752 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946e64c-dd62-4836-8cad-99bd99130cc8 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:11.967237 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946e64c-dd62-4836-8cad-99bd99130cc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:11.967716 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e946e64c-dd62-4836-8cad-99bd99130cc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:11.972073 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e946e64c-dd62-4836-8cad-99bd99130cc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:18:11.991586 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e946e64c-dd62-4836-8cad-99bd99130cc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:18:11.993313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e946e64c-dd62-4836-8cad-99bd99130cc8 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:18:12.013868 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1202/2451] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:18:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:18:19.671666 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=6 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:18:20.825230 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da1b1a1-48de-461b-a0da-e63318ada988 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:18:20.826580 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-3da1b1a1-48de-461b-a0da-e63318ada988 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:18:20.840826 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da1b1a1-48de-461b-a0da-e63318ada988 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:18:20.846476 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548256f8-acc2-4cf1-a2d9-f74091d87406 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:18:20.847113 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548256f8-acc2-4cf1-a2d9-f74091d87406 service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:20.847397 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548256f8-acc2-4cf1-a2d9-f74091d87406 service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:20.847672 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548256f8-acc2-4cf1-a2d9-f74091d87406 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:20.849443 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-548256f8-acc2-4cf1-a2d9-f74091d87406 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:18:20.859643 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548256f8-acc2-4cf1-a2d9-f74091d87406 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:18:20.861261 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548256f8-acc2-4cf1-a2d9-f74091d87406 service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:18:20.870706 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1250/2452] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:18:20 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:18:22.000794 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=31 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:18:30.865946 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=6 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:18:36.242188 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e946e64c-dd62-4836-8cad-99bd99130cc8 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:18:36.254079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e946e64c-dd62-4836-8cad-99bd99130cc8 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:18:36.258958 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f690cbd1-f886-431a-87ab-db530d86cac4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:18:36.260386 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690cbd1-f886-431a-87ab-db530d86cac4 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:36.261840 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690cbd1-f886-431a-87ab-db530d86cac4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:36.262442 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690cbd1-f886-431a-87ab-db530d86cac4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:36.265075 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-f690cbd1-f886-431a-87ab-db530d86cac4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:18:36.276812 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f690cbd1-f886-431a-87ab-db530d86cac4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:18:36.283890 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f690cbd1-f886-431a-87ab-db530d86cac4 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:18:36.290823 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1203/2453] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:18:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:18:36.448820 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548256f8-acc2-4cf1-a2d9-f74091d87406 service nova] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:18:36.460311 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548256f8-acc2-4cf1-a2d9-f74091d87406 service nova] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:18:36.464223 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67b9194-b23a-4503-83c9-7b8623fbc58e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:18:36.464899 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b9194-b23a-4503-83c9-7b8623fbc58e service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:36.465968 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b9194-b23a-4503-83c9-7b8623fbc58e service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:36.466572 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67b9194-b23a-4503-83c9-7b8623fbc58e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:36.468454 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b67b9194-b23a-4503-83c9-7b8623fbc58e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:18:36.555112 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67b9194-b23a-4503-83c9-7b8623fbc58e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:18:36.556634 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67b9194-b23a-4503-83c9-7b8623fbc58e service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:18:36.567643 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1251/2454] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:18:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:18:38.307603 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f690cbd1-f886-431a-87ab-db530d86cac4 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:18:38.321688 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f690cbd1-f886-431a-87ab-db530d86cac4 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:18:38.328332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065a4af1-3180-4aec-b22a-daee2f5e64bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:18:38.329099 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a4af1-3180-4aec-b22a-daee2f5e64bd service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:38.329369 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a4af1-3180-4aec-b22a-daee2f5e64bd service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:38.329646 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065a4af1-3180-4aec-b22a-daee2f5e64bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:38.331445 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-065a4af1-3180-4aec-b22a-daee2f5e64bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:18:38.345345 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065a4af1-3180-4aec-b22a-daee2f5e64bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:18:38.347480 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065a4af1-3180-4aec-b22a-daee2f5e64bd service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:18:38.377619 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1204/2455] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:18:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:18:41.177670 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3d64b-8c1d-45ab-93de-96e04730c0e1 None None] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:41.178225 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3d64b-8c1d-45ab-93de-96e04730c0e1 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:41.178435 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe3d64b-8c1d-45ab-93de-96e04730c0e1 None None] PATH_INFO: `/` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:41.180150 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1252/2456] 10.222.0.40 () {54 vars in 921 bytes} [Wed Jul 6 17:18:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:18:41.193781 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0495d2-a0fa-49f4-9960-a137f49941ce None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:41.193781 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0495d2-a0fa-49f4-9960-a137f49941ce None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:41.193781 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0495d2-a0fa-49f4-9960-a137f49941ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:41.230402 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-9b0495d2-a0fa-49f4-9960-a137f49941ce None None] MFA Rules not processed for user `76ca877bee9041d28ad41e45ef3dd18c`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:18:41.230402 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-9b0495d2-a0fa-49f4-9960-a137f49941ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:18:41.238853 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1205/2457] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Jul 6 17:18:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:18:41.251335 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe3d64b-8c1d-45ab-93de-96e04730c0e1 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:18:41.266273 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe3d64b-8c1d-45ab-93de-96e04730c0e1 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:18:41.271219 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cbf9e5-17c9-4245-a87f-e1cc17df1168 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76ca877bee9041d28ad41e45ef3dd18c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:18:41.271952 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cbf9e5-17c9-4245-a87f-e1cc17df1168 service placement] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:41.272212 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cbf9e5-17c9-4245-a87f-e1cc17df1168 service placement] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:41.272487 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cbf9e5-17c9-4245-a87f-e1cc17df1168 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:41.281251 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cbf9e5-17c9-4245-a87f-e1cc17df1168 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:18:41.282835 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cbf9e5-17c9-4245-a87f-e1cc17df1168 service placement] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:18:41.292639 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1253/2458] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:18:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:18:41.346412 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0495d2-a0fa-49f4-9960-a137f49941ce None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:18:41.347900 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-9b0495d2-a0fa-49f4-9960-a137f49941ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:18:41.362190 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0495d2-a0fa-49f4-9960-a137f49941ce None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:18:41.373343 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f98ebc2-97e1-4fdf-a26b-2414c3ea3efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:18:41.373343 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98ebc2-97e1-4fdf-a26b-2414c3ea3efd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:41.373343 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98ebc2-97e1-4fdf-a26b-2414c3ea3efd admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:41.373343 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f98ebc2-97e1-4fdf-a26b-2414c3ea3efd admin admin] PATH_INFO: `/v3/users/8e302cf8d56c4db8bb8480ec234ac908` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:41.373735 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f98ebc2-97e1-4fdf-a26b-2414c3ea3efd admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e302cf8d56c4db8bb8480ec234ac908)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:18:41.373735 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f98ebc2-97e1-4fdf-a26b-2414c3ea3efd admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:18:41.578273 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0f98ebc2-97e1-4fdf-a26b-2414c3ea3efd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e302cf8d56c4db8bb8480ec234ac908'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:18:41.589915 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0f98ebc2-97e1-4fdf-a26b-2414c3ea3efd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e302cf8d56c4db8bb8480ec234ac908'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:18:41.597349 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0f98ebc2-97e1-4fdf-a26b-2414c3ea3efd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e302cf8d56c4db8bb8480ec234ac908'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:18:41.598035 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0f98ebc2-97e1-4fdf-a26b-2414c3ea3efd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e302cf8d56c4db8bb8480ec234ac908'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:18:41.656282 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0f98ebc2-97e1-4fdf-a26b-2414c3ea3efd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e302cf8d56c4db8bb8480ec234ac908'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:18:41.662567 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1206/2459] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:18:41 2022] DELETE /identity/v3/users/8e302cf8d56c4db8bb8480ec234ac908 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:18:41.942936 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cbf9e5-17c9-4245-a87f-e1cc17df1168 service placement] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:18:41.956076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cbf9e5-17c9-4245-a87f-e1cc17df1168 service placement] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:18:42.013582 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec68b64-38c7-4bb6-99bd-0425f88487c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:18:42.014435 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec68b64-38c7-4bb6-99bd-0425f88487c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:42.014713 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec68b64-38c7-4bb6-99bd-0425f88487c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:42.014984 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec68b64-38c7-4bb6-99bd-0425f88487c9 admin admin] PATH_INFO: `/v3/projects/13a0db3fcd71427e9a442c589f435eb4` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:42.016182 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec68b64-38c7-4bb6-99bd-0425f88487c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=13a0db3fcd71427e9a442c589f435eb4)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:18:42.018094 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec68b64-38c7-4bb6-99bd-0425f88487c9 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:18:42.131015 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-eec68b64-38c7-4bb6-99bd-0425f88487c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '13a0db3fcd71427e9a442c589f435eb4', 'request_id': 'req-eec68b64-38c7-4bb6-99bd-0425f88487c9'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:18:42.137787 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-eec68b64-38c7-4bb6-99bd-0425f88487c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '13a0db3fcd71427e9a442c589f435eb4', 'request_id': 'req-eec68b64-38c7-4bb6-99bd-0425f88487c9'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:18:42.143207 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-eec68b64-38c7-4bb6-99bd-0425f88487c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '13a0db3fcd71427e9a442c589f435eb4', 'request_id': 'req-eec68b64-38c7-4bb6-99bd-0425f88487c9'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:18:42.236082 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-eec68b64-38c7-4bb6-99bd-0425f88487c9 admin admin] The token cache is being invalidate because project 13a0db3fcd71427e9a442c589f435eb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:18:42.236685 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-eec68b64-38c7-4bb6-99bd-0425f88487c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:18:42.236896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-eec68b64-38c7-4bb6-99bd-0425f88487c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:18:42.238564 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1254/2460] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:18:41 2022] DELETE /identity/v3/projects/13a0db3fcd71427e9a442c589f435eb4 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:18:42.243503 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff7ce1-837d-42c9-8263-630829d9cdcb None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:42.244016 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff7ce1-837d-42c9-8263-630829d9cdcb None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:42.244120 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adff7ce1-837d-42c9-8263-630829d9cdcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:42.262793 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-adff7ce1-837d-42c9-8263-630829d9cdcb None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:18:42.264017 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-adff7ce1-837d-42c9-8263-630829d9cdcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:18:42.325002 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1207/2461] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:18:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:18:42.333272 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80afad04-8e18-4594-b4de-c5e2d8b047c2 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:42.348451 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80afad04-8e18-4594-b4de-c5e2d8b047c2 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:42.348451 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80afad04-8e18-4594-b4de-c5e2d8b047c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:42.352565 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-80afad04-8e18-4594-b4de-c5e2d8b047c2 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:18:42.353764 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-80afad04-8e18-4594-b4de-c5e2d8b047c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:18:42.361674 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1255/2462] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:18:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:18:42.365927 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adff7ce1-837d-42c9-8263-630829d9cdcb None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:18:42.375959 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adff7ce1-837d-42c9-8263-630829d9cdcb None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:18:42.380618 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a21ce93-5a5f-4bf8-8140-f4c3c066470e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:18:42.381272 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a21ce93-5a5f-4bf8-8140-f4c3c066470e admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:42.381555 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a21ce93-5a5f-4bf8-8140-f4c3c066470e admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:42.381829 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a21ce93-5a5f-4bf8-8140-f4c3c066470e admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:42.382687 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a21ce93-5a5f-4bf8-8140-f4c3c066470e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:18:42.382928 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a21ce93-5a5f-4bf8-8140-f4c3c066470e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:18:42.384655 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a21ce93-5a5f-4bf8-8140-f4c3c066470e admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:18:42.396820 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1208/2463] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:18:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:18:42.401067 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80afad04-8e18-4594-b4de-c5e2d8b047c2 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:18:42.407549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:18:42.411202 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80afad04-8e18-4594-b4de-c5e2d8b047c2 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:18:42.415998 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b3a65e-9956-44e7-872c-42a2808cd502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:18:42.416667 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3a65e-9956-44e7-872c-42a2808cd502 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:42.416948 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3a65e-9956-44e7-872c-42a2808cd502 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:42.417222 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3a65e-9956-44e7-872c-42a2808cd502 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:42.417939 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b3a65e-9956-44e7-872c-42a2808cd502 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:18:42.419923 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b3a65e-9956-44e7-872c-42a2808cd502 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:18:42.526397 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1256/2464] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:18:42 2022] POST /identity/v3/projects => generated 373 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:18:42.531496 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a21ce93-5a5f-4bf8-8140-f4c3c066470e admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:18:42.541497 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a21ce93-5a5f-4bf8-8140-f4c3c066470e admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:18:42.550056 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:18:42.596089 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c4da9-b8e2-4d44-81ee-db12744a5b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:18:42.596750 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c4da9-b8e2-4d44-81ee-db12744a5b20 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:42.597034 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c4da9-b8e2-4d44-81ee-db12744a5b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:42.597309 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5c4da9-b8e2-4d44-81ee-db12744a5b20 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:42.598104 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5c4da9-b8e2-4d44-81ee-db12744a5b20 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:18:42.599951 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5c4da9-b8e2-4d44-81ee-db12744a5b20 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:18:42.723520 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1209/2465] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:18:42 2022] POST /identity/v3/users => generated 287 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:18:42.727476 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b3a65e-9956-44e7-872c-42a2808cd502 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:18:42.741489 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b3a65e-9956-44e7-872c-42a2808cd502 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:18:42.746858 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92321bb-fbf7-440b-918f-0cc74ea452f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:18:42.747535 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92321bb-fbf7-440b-918f-0cc74ea452f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:42.747922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92321bb-fbf7-440b-918f-0cc74ea452f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:42.748202 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92321bb-fbf7-440b-918f-0cc74ea452f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:42.748938 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92321bb-fbf7-440b-918f-0cc74ea452f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:18:42.750638 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92321bb-fbf7-440b-918f-0cc74ea452f0 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:18:42.764261 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1257/2466] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:18:42.765579 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c4da9-b8e2-4d44-81ee-db12744a5b20 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:18:42.775939 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5c4da9-b8e2-4d44-81ee-db12744a5b20 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:18:42.781657 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8b2aa3-e969-4012-9763-c07b9d4b8c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:18:42.782298 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b2aa3-e969-4012-9763-c07b9d4b8c0f admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:42.782584 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b2aa3-e969-4012-9763-c07b9d4b8c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:42.782868 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8b2aa3-e969-4012-9763-c07b9d4b8c0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:42.783598 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8b2aa3-e969-4012-9763-c07b9d4b8c0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:18:42.785345 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8b2aa3-e969-4012-9763-c07b9d4b8c0f admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:18:42.796660 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1210/2467] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:18:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:18:42.802519 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92321bb-fbf7-440b-918f-0cc74ea452f0 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:18:42.813138 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92321bb-fbf7-440b-918f-0cc74ea452f0 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:18:42.817763 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4249fd-3e4f-41ce-9435-19c658ed6854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:18:42.818644 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4249fd-3e4f-41ce-9435-19c658ed6854 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:42.818926 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4249fd-3e4f-41ce-9435-19c658ed6854 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:42.819203 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4249fd-3e4f-41ce-9435-19c658ed6854 admin admin] PATH_INFO: `/v3/projects/6718d1451ac54145b3f1eee000f25c7a/users/fbda5b7ddf9343439c1fc0efc2e69557/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:42.835271 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4249fd-3e4f-41ce-9435-19c658ed6854 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6718d1451ac54145b3f1eee000f25c7a, user_id=fbda5b7ddf9343439c1fc0efc2e69557, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:18:42.838021 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4249fd-3e4f-41ce-9435-19c658ed6854 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:18:42.896787 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1258/2468] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:18:42 2022] PUT /identity/v3/projects/6718d1451ac54145b3f1eee000f25c7a/users/fbda5b7ddf9343439c1fc0efc2e69557/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:18:42.907202 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa5eac9-69be-43b7-af74-dd7cd76fb4e1 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:42.907501 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa5eac9-69be-43b7-af74-dd7cd76fb4e1 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:42.907807 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa5eac9-69be-43b7-af74-dd7cd76fb4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:42.934787 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-bfa5eac9-69be-43b7-af74-dd7cd76fb4e1 None None] MFA Rules not processed for user `fbda5b7ddf9343439c1fc0efc2e69557`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:18:42.936025 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-bfa5eac9-69be-43b7-af74-dd7cd76fb4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:18:42.998728 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1211/2469] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:18:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:18:43.010199 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4249fd-3e4f-41ce-9435-19c658ed6854 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:18:43.018731 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-bf4249fd-3e4f-41ce-9435-19c658ed6854 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:18:43.084264 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4249fd-3e4f-41ce-9435-19c658ed6854 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:18:43.089844 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909294cb-e045-4d36-bd44-193275a482de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:18:43.090518 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909294cb-e045-4d36-bd44-193275a482de service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:43.090809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909294cb-e045-4d36-bd44-193275a482de service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:43.091087 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909294cb-e045-4d36-bd44-193275a482de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:43.098910 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909294cb-e045-4d36-bd44-193275a482de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:18:43.100535 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909294cb-e045-4d36-bd44-193275a482de service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:18:43.109717 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1259/2470] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:18:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:18:45.170242 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa5eac9-69be-43b7-af74-dd7cd76fb4e1 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:18:45.182676 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa5eac9-69be-43b7-af74-dd7cd76fb4e1 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:18:45.188112 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ee3695-1e83-4cda-83e1-6d35790ee1c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:18:45.188746 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee3695-1e83-4cda-83e1-6d35790ee1c2 service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:45.189749 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee3695-1e83-4cda-83e1-6d35790ee1c2 service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:45.189749 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee3695-1e83-4cda-83e1-6d35790ee1c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:45.191188 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-72ee3695-1e83-4cda-83e1-6d35790ee1c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:18:45.203840 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ee3695-1e83-4cda-83e1-6d35790ee1c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:18:45.205409 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ee3695-1e83-4cda-83e1-6d35790ee1c2 service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:18:45.218271 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1212/2471] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:18:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:18:46.339293 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909294cb-e045-4d36-bd44-193275a482de service nova] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:18:46.364245 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b513822-d629-4d0b-a96f-504657b12cae None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:46.364812 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b513822-d629-4d0b-a96f-504657b12cae None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:46.364812 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b513822-d629-4d0b-a96f-504657b12cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:46.382504 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-2b513822-d629-4d0b-a96f-504657b12cae None None] MFA Rules not processed for user `1905e6f8c3e24e3cbfefce2827baa370`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:18:46.383717 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-2b513822-d629-4d0b-a96f-504657b12cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:18:46.407546 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909294cb-e045-4d36-bd44-193275a482de service nova] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:18:46.412575 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a272abad-1139-4118-bfd2-6653c01a4426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:18:46.413444 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a272abad-1139-4118-bfd2-6653c01a4426 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:46.413733 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a272abad-1139-4118-bfd2-6653c01a4426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:46.414009 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a272abad-1139-4118-bfd2-6653c01a4426 admin admin] PATH_INFO: `/v3/users/fbda5b7ddf9343439c1fc0efc2e69557` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:46.415207 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a272abad-1139-4118-bfd2-6653c01a4426 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbda5b7ddf9343439c1fc0efc2e69557)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:18:46.417152 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a272abad-1139-4118-bfd2-6653c01a4426 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:18:46.441395 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1213/2472] 10.222.0.40 () {58 vars in 967 bytes} [Wed Jul 6 17:18:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:18:46.592961 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b513822-d629-4d0b-a96f-504657b12cae None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:18:46.594386 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-2b513822-d629-4d0b-a96f-504657b12cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:18:46.612287 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:18:46.662092 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:18:46.672697 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-a272abad-1139-4118-bfd2-6653c01a4426 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbda5b7ddf9343439c1fc0efc2e69557'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:18:46.678368 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b513822-d629-4d0b-a96f-504657b12cae None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:18:46.680001 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-a272abad-1139-4118-bfd2-6653c01a4426 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbda5b7ddf9343439c1fc0efc2e69557'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:18:46.680667 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-a272abad-1139-4118-bfd2-6653c01a4426 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbda5b7ddf9343439c1fc0efc2e69557'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:18:46.684146 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f794e38-fd50-4536-ba63-1ab59ddd08d6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75acacd30e9a4cf2ad58b072b49e3f05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:18:46.684958 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f794e38-fd50-4536-ba63-1ab59ddd08d6 service swift] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:46.685697 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f794e38-fd50-4536-ba63-1ab59ddd08d6 service swift] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:46.686080 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f794e38-fd50-4536-ba63-1ab59ddd08d6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:46.692736 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-a272abad-1139-4118-bfd2-6653c01a4426 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbda5b7ddf9343439c1fc0efc2e69557'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:18:46.694847 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f794e38-fd50-4536-ba63-1ab59ddd08d6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:18:46.696480 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f794e38-fd50-4536-ba63-1ab59ddd08d6 service swift] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:18:46.704993 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1214/2473] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 6 17:18:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 06 17:18:46.764661 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-a272abad-1139-4118-bfd2-6653c01a4426 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbda5b7ddf9343439c1fc0efc2e69557'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:18:46.771212 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1260/2474] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:18:46 2022] DELETE /identity/v3/users/fbda5b7ddf9343439c1fc0efc2e69557 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:18:46.775677 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ac037-2077-4443-836e-532602f8ca3d None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:46.775999 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ac037-2077-4443-836e-532602f8ca3d None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:46.776277 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6ac037-2077-4443-836e-532602f8ca3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:46.795205 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-2b6ac037-2077-4443-836e-532602f8ca3d None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:18:46.796454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-2b6ac037-2077-4443-836e-532602f8ca3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:18:46.854200 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1215/2475] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:18:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:18:46.863613 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a272abad-1139-4118-bfd2-6653c01a4426 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:18:46.865047 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-a272abad-1139-4118-bfd2-6653c01a4426 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:18:46.925280 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a272abad-1139-4118-bfd2-6653c01a4426 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:18:46.929042 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f95d15-14da-435e-aea2-02f88ff33d2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:18:46.929695 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f95d15-14da-435e-aea2-02f88ff33d2a service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:46.929986 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f95d15-14da-435e-aea2-02f88ff33d2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:46.930266 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f95d15-14da-435e-aea2-02f88ff33d2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:46.939270 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f95d15-14da-435e-aea2-02f88ff33d2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:18:46.940737 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f95d15-14da-435e-aea2-02f88ff33d2a service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:18:46.950878 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1261/2476] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:18:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:18:47.554730 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6ac037-2077-4443-836e-532602f8ca3d None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:18:47.558551 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-2b6ac037-2077-4443-836e-532602f8ca3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:18:47.570446 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6ac037-2077-4443-836e-532602f8ca3d None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:18:47.576134 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17722db1-b342-4451-b48f-206212514c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:18:47.577017 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17722db1-b342-4451-b48f-206212514c0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:47.577305 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17722db1-b342-4451-b48f-206212514c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:47.577583 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17722db1-b342-4451-b48f-206212514c0d admin admin] PATH_INFO: `/v3/projects/6718d1451ac54145b3f1eee000f25c7a` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:47.579279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17722db1-b342-4451-b48f-206212514c0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6718d1451ac54145b3f1eee000f25c7a)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:18:47.581568 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17722db1-b342-4451-b48f-206212514c0d admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:18:47.697688 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-17722db1-b342-4451-b48f-206212514c0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6718d1451ac54145b3f1eee000f25c7a', 'request_id': 'req-17722db1-b342-4451-b48f-206212514c0d'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:18:47.704408 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-17722db1-b342-4451-b48f-206212514c0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6718d1451ac54145b3f1eee000f25c7a', 'request_id': 'req-17722db1-b342-4451-b48f-206212514c0d'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:18:47.770309 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-17722db1-b342-4451-b48f-206212514c0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6718d1451ac54145b3f1eee000f25c7a', 'request_id': 'req-17722db1-b342-4451-b48f-206212514c0d'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:18:47.782082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-17722db1-b342-4451-b48f-206212514c0d admin admin] The token cache is being invalidate because project 6718d1451ac54145b3f1eee000f25c7a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:18:47.782381 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-17722db1-b342-4451-b48f-206212514c0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:18:47.783108 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-17722db1-b342-4451-b48f-206212514c0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:18:47.784805 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1216/2477] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:18:47 2022] DELETE /identity/v3/projects/6718d1451ac54145b3f1eee000f25c7a => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:18:47.789796 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a3d41-69a4-4a0a-88d3-d41afdd6020d None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:47.790260 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a3d41-69a4-4a0a-88d3-d41afdd6020d None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:47.790533 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a3d41-69a4-4a0a-88d3-d41afdd6020d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:47.811002 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-410a3d41-69a4-4a0a-88d3-d41afdd6020d None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:18:47.812178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-410a3d41-69a4-4a0a-88d3-d41afdd6020d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:18:47.883893 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1262/2478] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:18:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:18:47.888782 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5a1e6-84ac-42f4-b0cc-eee0d82230b7 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:47.889230 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5a1e6-84ac-42f4-b0cc-eee0d82230b7 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:47.890278 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d5a1e6-84ac-42f4-b0cc-eee0d82230b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:47.909881 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-f5d5a1e6-84ac-42f4-b0cc-eee0d82230b7 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:18:47.911079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-f5d5a1e6-84ac-42f4-b0cc-eee0d82230b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:18:47.920286 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1217/2479] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:18:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:18:47.924513 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410a3d41-69a4-4a0a-88d3-d41afdd6020d None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:18:47.934400 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410a3d41-69a4-4a0a-88d3-d41afdd6020d None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:18:47.939425 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abaf816-bd8b-4ece-a77f-a3e4335baea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:18:47.940117 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abaf816-bd8b-4ece-a77f-a3e4335baea2 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:47.940406 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abaf816-bd8b-4ece-a77f-a3e4335baea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:47.940689 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abaf816-bd8b-4ece-a77f-a3e4335baea2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:47.941424 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abaf816-bd8b-4ece-a77f-a3e4335baea2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:18:47.941736 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abaf816-bd8b-4ece-a77f-a3e4335baea2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:18:47.943363 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abaf816-bd8b-4ece-a77f-a3e4335baea2 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:18:47.958966 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1263/2480] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:18:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:18:47.960363 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d5a1e6-84ac-42f4-b0cc-eee0d82230b7 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:18:47.970441 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d5a1e6-84ac-42f4-b0cc-eee0d82230b7 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:18:47.975188 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf32b9-def5-4728-865d-79bb3ba2b92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:18:47.975877 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf32b9-def5-4728-865d-79bb3ba2b92f admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:47.976160 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf32b9-def5-4728-865d-79bb3ba2b92f admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:47.976439 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cf32b9-def5-4728-865d-79bb3ba2b92f admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:47.977157 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf32b9-def5-4728-865d-79bb3ba2b92f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:18:47.979094 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cf32b9-def5-4728-865d-79bb3ba2b92f admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:18:47.983909 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:18:48.076164 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1218/2481] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:18:47 2022] POST /identity/v3/projects => generated 381 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:18:48.080222 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abaf816-bd8b-4ece-a77f-a3e4335baea2 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:18:48.090372 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abaf816-bd8b-4ece-a77f-a3e4335baea2 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:18:48.133727 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:18:48.150625 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2959e5c9-090b-4857-8c0f-453e76c9741a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:18:48.151307 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959e5c9-090b-4857-8c0f-453e76c9741a admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:48.151595 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959e5c9-090b-4857-8c0f-453e76c9741a admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:48.151910 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959e5c9-090b-4857-8c0f-453e76c9741a admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:48.152620 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2959e5c9-090b-4857-8c0f-453e76c9741a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:18:48.154901 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2959e5c9-090b-4857-8c0f-453e76c9741a admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:18:48.228204 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1264/2482] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:18:48 2022] POST /identity/v3/users => generated 291 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:18:48.233042 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf32b9-def5-4728-865d-79bb3ba2b92f admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:18:48.243709 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cf32b9-def5-4728-865d-79bb3ba2b92f admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:18:48.249258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed22ca0b-df18-437b-9369-36e09af07b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:18:48.251602 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22ca0b-df18-437b-9369-36e09af07b37 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:48.251602 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22ca0b-df18-437b-9369-36e09af07b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:48.251602 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed22ca0b-df18-437b-9369-36e09af07b37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:48.251602 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed22ca0b-df18-437b-9369-36e09af07b37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:18:48.253262 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed22ca0b-df18-437b-9369-36e09af07b37 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:18:48.263652 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1219/2483] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:18:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:18:48.268611 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2959e5c9-090b-4857-8c0f-453e76c9741a admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:18:48.279208 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2959e5c9-090b-4857-8c0f-453e76c9741a admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:18:48.284702 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6c664a-b53b-460a-9b89-89fd28e42e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:18:48.285390 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6c664a-b53b-460a-9b89-89fd28e42e12 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:48.285663 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6c664a-b53b-460a-9b89-89fd28e42e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:48.285941 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6c664a-b53b-460a-9b89-89fd28e42e12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:48.286671 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6c664a-b53b-460a-9b89-89fd28e42e12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:18:48.288295 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6c664a-b53b-460a-9b89-89fd28e42e12 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:18:48.302414 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1265/2484] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:18:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:18:48.303887 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed22ca0b-df18-437b-9369-36e09af07b37 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:18:48.314665 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed22ca0b-df18-437b-9369-36e09af07b37 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:18:48.319600 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4752b5ce-55a5-4b43-8eb3-e721a41a0ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:18:48.320520 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752b5ce-55a5-4b43-8eb3-e721a41a0ab5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:48.320801 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752b5ce-55a5-4b43-8eb3-e721a41a0ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:48.321077 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752b5ce-55a5-4b43-8eb3-e721a41a0ab5 admin admin] PATH_INFO: `/v3/projects/a2e9151e312a4aa7a35553c029c99276/users/bf57fc11325d46ac80b5b74ccf1d152d/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:48.336317 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4752b5ce-55a5-4b43-8eb3-e721a41a0ab5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2e9151e312a4aa7a35553c029c99276, user_id=bf57fc11325d46ac80b5b74ccf1d152d, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:18:48.339040 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4752b5ce-55a5-4b43-8eb3-e721a41a0ab5 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:18:48.426574 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1220/2485] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:18:48 2022] PUT /identity/v3/projects/a2e9151e312a4aa7a35553c029c99276/users/bf57fc11325d46ac80b5b74ccf1d152d/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:18:48.437973 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8fbce0-cbfd-467a-b31f-41ecb3716c3c None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:48.438268 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8fbce0-cbfd-467a-b31f-41ecb3716c3c None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:48.438550 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8fbce0-cbfd-467a-b31f-41ecb3716c3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:48.466661 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-1c8fbce0-cbfd-467a-b31f-41ecb3716c3c None None] MFA Rules not processed for user `bf57fc11325d46ac80b5b74ccf1d152d`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:18:48.468207 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-1c8fbce0-cbfd-467a-b31f-41ecb3716c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:18:48.536846 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1266/2486] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:18:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:18:48.541243 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4752b5ce-55a5-4b43-8eb3-e721a41a0ab5 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:18:48.604502 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4752b5ce-55a5-4b43-8eb3-e721a41a0ab5 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:18:48.609335 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf97565-bd6b-493f-931c-a8c541123725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:18:48.610050 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf97565-bd6b-493f-931c-a8c541123725 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:48.615667 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf97565-bd6b-493f-931c-a8c541123725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:48.615667 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf97565-bd6b-493f-931c-a8c541123725 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:48.615667 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf97565-bd6b-493f-931c-a8c541123725 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:18:48.615667 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf97565-bd6b-493f-931c-a8c541123725 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:18:48.710461 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1221/2487] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:18:48 2022] POST /identity/v3/projects => generated 381 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:18:48.711691 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8fbce0-cbfd-467a-b31f-41ecb3716c3c None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:18:48.722915 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8fbce0-cbfd-467a-b31f-41ecb3716c3c None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:18:48.783545 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a520c7-13dd-4b9e-aee5-359843116f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:18:48.784304 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a520c7-13dd-4b9e-aee5-359843116f67 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:48.784593 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a520c7-13dd-4b9e-aee5-359843116f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:48.784900 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a520c7-13dd-4b9e-aee5-359843116f67 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:48.785599 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a520c7-13dd-4b9e-aee5-359843116f67 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:18:48.787579 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a520c7-13dd-4b9e-aee5-359843116f67 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:18:48.915788 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1267/2488] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:18:48 2022] POST /identity/v3/users => generated 297 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:18:48.917657 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf97565-bd6b-493f-931c-a8c541123725 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:18:48.929073 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf97565-bd6b-493f-931c-a8c541123725 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:18:48.935509 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305109b3-5e15-4a0d-bd3f-e88ec398964e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:18:48.935897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305109b3-5e15-4a0d-bd3f-e88ec398964e admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:48.936177 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305109b3-5e15-4a0d-bd3f-e88ec398964e admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:48.936455 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305109b3-5e15-4a0d-bd3f-e88ec398964e admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:48.937298 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305109b3-5e15-4a0d-bd3f-e88ec398964e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:18:48.938969 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305109b3-5e15-4a0d-bd3f-e88ec398964e admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:18:48.955383 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1222/2489] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:18:48 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:18:48.956616 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a520c7-13dd-4b9e-aee5-359843116f67 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:18:48.968011 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a520c7-13dd-4b9e-aee5-359843116f67 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:18:48.973308 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da37c7a-88f3-4bf2-b1c6-d4521062e5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:18:48.974374 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da37c7a-88f3-4bf2-b1c6-d4521062e5ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:48.974730 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da37c7a-88f3-4bf2-b1c6-d4521062e5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:48.975072 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da37c7a-88f3-4bf2-b1c6-d4521062e5ec admin admin] PATH_INFO: `/v3/projects/04f4d22d65ba42d194d937c01344c395/users/126aeecbc9be4a33a586d236655dd6e1/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:48.993680 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da37c7a-88f3-4bf2-b1c6-d4521062e5ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=04f4d22d65ba42d194d937c01344c395, user_id=126aeecbc9be4a33a586d236655dd6e1, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:18:48.997246 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da37c7a-88f3-4bf2-b1c6-d4521062e5ec admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:18:49.083767 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1268/2490] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:18:48 2022] PUT /identity/v3/projects/04f4d22d65ba42d194d937c01344c395/users/126aeecbc9be4a33a586d236655dd6e1/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:18:49.087916 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305109b3-5e15-4a0d-bd3f-e88ec398964e admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:18:49.098384 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305109b3-5e15-4a0d-bd3f-e88ec398964e admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:18:49.157781 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f110fa-46b9-498a-b319-1a8ac71efd58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:18:49.158474 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f110fa-46b9-498a-b319-1a8ac71efd58 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:49.158760 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f110fa-46b9-498a-b319-1a8ac71efd58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:49.159041 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f110fa-46b9-498a-b319-1a8ac71efd58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:49.159820 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f110fa-46b9-498a-b319-1a8ac71efd58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:18:49.161485 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f110fa-46b9-498a-b319-1a8ac71efd58 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:18:49.177252 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1223/2491] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:18:49.181360 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da37c7a-88f3-4bf2-b1c6-d4521062e5ec admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:18:49.192516 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da37c7a-88f3-4bf2-b1c6-d4521062e5ec admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:18:49.197450 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161191d-d256-414f-bd33-b52dd79ed426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:18:49.198346 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161191d-d256-414f-bd33-b52dd79ed426 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:49.198625 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161191d-d256-414f-bd33-b52dd79ed426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:49.198901 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f161191d-d256-414f-bd33-b52dd79ed426 admin admin] PATH_INFO: `/v3/projects/04f4d22d65ba42d194d937c01344c395/users/126aeecbc9be4a33a586d236655dd6e1/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:49.200928 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f161191d-d256-414f-bd33-b52dd79ed426 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04f4d22d65ba42d194d937c01344c395, user_id=126aeecbc9be4a33a586d236655dd6e1, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:18:49.203560 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f161191d-d256-414f-bd33-b52dd79ed426 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:18:49.214286 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1269/2492] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:18:49 2022] PUT /identity/v3/projects/04f4d22d65ba42d194d937c01344c395/users/126aeecbc9be4a33a586d236655dd6e1/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:18:49.224481 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7441dbb-60a4-4aed-a4b2-199679411a79 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:49.224778 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7441dbb-60a4-4aed-a4b2-199679411a79 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:49.225058 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7441dbb-60a4-4aed-a4b2-199679411a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:49.253446 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-a7441dbb-60a4-4aed-a4b2-199679411a79 None None] MFA Rules not processed for user `126aeecbc9be4a33a586d236655dd6e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:18:49.254715 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-a7441dbb-60a4-4aed-a4b2-199679411a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:18:49.265360 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:18:49.325771 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1224/2493] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:18:49 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:18:49.337721 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161191d-d256-414f-bd33-b52dd79ed426 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:18:49.339714 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f161191d-d256-414f-bd33-b52dd79ed426 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:18:49.344613 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:18:49.423491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f161191d-d256-414f-bd33-b52dd79ed426 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:18:49.428320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12aa5d0-1c54-4dc3-9cea-41d81d97284e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:18:49.428320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12aa5d0-1c54-4dc3-9cea-41d81d97284e service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:49.428320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12aa5d0-1c54-4dc3-9cea-41d81d97284e service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:49.428619 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12aa5d0-1c54-4dc3-9cea-41d81d97284e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:49.438559 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12aa5d0-1c54-4dc3-9cea-41d81d97284e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:18:49.440035 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12aa5d0-1c54-4dc3-9cea-41d81d97284e service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:18:49.450726 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1270/2494] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:18:49 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:18:49.645640 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7441dbb-60a4-4aed-a4b2-199679411a79 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:18:49.656843 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7441dbb-60a4-4aed-a4b2-199679411a79 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:18:49.660580 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0af3c2-f69f-4585-bdb7-0dc07d29f4df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:18:49.665084 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0af3c2-f69f-4585-bdb7-0dc07d29f4df service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:49.665084 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0af3c2-f69f-4585-bdb7-0dc07d29f4df service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:49.665084 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0af3c2-f69f-4585-bdb7-0dc07d29f4df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:49.737114 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0af3c2-f69f-4585-bdb7-0dc07d29f4df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:18:49.739605 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0af3c2-f69f-4585-bdb7-0dc07d29f4df service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:18:49.752764 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1225/2495] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:18:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:18:55.521495 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12aa5d0-1c54-4dc3-9cea-41d81d97284e service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:18:55.522912 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c12aa5d0-1c54-4dc3-9cea-41d81d97284e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:18:55.594458 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12aa5d0-1c54-4dc3-9cea-41d81d97284e service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:18:55.600159 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04714c30-539d-4ce5-a933-9e0754640eed service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76ca877bee9041d28ad41e45ef3dd18c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:18:55.600822 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04714c30-539d-4ce5-a933-9e0754640eed service placement] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:55.601107 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04714c30-539d-4ce5-a933-9e0754640eed service placement] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:55.601386 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04714c30-539d-4ce5-a933-9e0754640eed service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:55.603566 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-04714c30-539d-4ce5-a933-9e0754640eed service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:18:55.618330 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04714c30-539d-4ce5-a933-9e0754640eed service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:18:55.619989 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04714c30-539d-4ce5-a933-9e0754640eed service placement] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:18:55.630846 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1271/2496] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:18:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:18:55.644610 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0af3c2-f69f-4585-bdb7-0dc07d29f4df service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:18:55.646193 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-2d0af3c2-f69f-4585-bdb7-0dc07d29f4df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:18:55.717163 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0af3c2-f69f-4585-bdb7-0dc07d29f4df service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:18:55.721554 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ff82f-49dc-495a-a391-6ddf8b1c4891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:18:55.722565 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ff82f-49dc-495a-a391-6ddf8b1c4891 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:55.722883 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ff82f-49dc-495a-a391-6ddf8b1c4891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:55.723176 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20ff82f-49dc-495a-a391-6ddf8b1c4891 admin admin] PATH_INFO: `/v3/users/43974c97019b47908a00fc7261fc22a2` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:55.724563 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20ff82f-49dc-495a-a391-6ddf8b1c4891 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43974c97019b47908a00fc7261fc22a2)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:18:55.726485 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20ff82f-49dc-495a-a391-6ddf8b1c4891 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:18:56.012341 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d20ff82f-49dc-495a-a391-6ddf8b1c4891 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43974c97019b47908a00fc7261fc22a2'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:18:56.013471 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:18:56.024413 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d20ff82f-49dc-495a-a391-6ddf8b1c4891 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43974c97019b47908a00fc7261fc22a2'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:18:56.031282 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d20ff82f-49dc-495a-a391-6ddf8b1c4891 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43974c97019b47908a00fc7261fc22a2'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:18:56.031976 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d20ff82f-49dc-495a-a391-6ddf8b1c4891 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43974c97019b47908a00fc7261fc22a2'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:18:56.147575 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-d20ff82f-49dc-495a-a391-6ddf8b1c4891 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43974c97019b47908a00fc7261fc22a2'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:18:56.154417 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1226/2497] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:18:55 2022] DELETE /identity/v3/users/43974c97019b47908a00fc7261fc22a2 => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:18:56.415591 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04714c30-539d-4ce5-a933-9e0754640eed service placement] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:18:56.427041 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04714c30-539d-4ce5-a933-9e0754640eed service placement] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:18:56.487622 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d72dbd-65f8-46ed-a898-b37342ed4a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:18:56.488511 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d72dbd-65f8-46ed-a898-b37342ed4a36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:56.488800 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d72dbd-65f8-46ed-a898-b37342ed4a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:56.489058 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d72dbd-65f8-46ed-a898-b37342ed4a36 admin admin] PATH_INFO: `/v3/projects/966f22e2bdf44625a350f3647b281d70` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:56.490283 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d72dbd-65f8-46ed-a898-b37342ed4a36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=966f22e2bdf44625a350f3647b281d70)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:18:56.492216 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d72dbd-65f8-46ed-a898-b37342ed4a36 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:18:56.508559 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:18:56.630401 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-70d72dbd-65f8-46ed-a898-b37342ed4a36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '966f22e2bdf44625a350f3647b281d70', 'request_id': 'req-70d72dbd-65f8-46ed-a898-b37342ed4a36'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:18:56.636420 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-70d72dbd-65f8-46ed-a898-b37342ed4a36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '966f22e2bdf44625a350f3647b281d70', 'request_id': 'req-70d72dbd-65f8-46ed-a898-b37342ed4a36'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:18:56.642456 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-70d72dbd-65f8-46ed-a898-b37342ed4a36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '966f22e2bdf44625a350f3647b281d70', 'request_id': 'req-70d72dbd-65f8-46ed-a898-b37342ed4a36'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:18:56.730967 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-70d72dbd-65f8-46ed-a898-b37342ed4a36 admin admin] The token cache is being invalidate because project 966f22e2bdf44625a350f3647b281d70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:18:56.731307 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-70d72dbd-65f8-46ed-a898-b37342ed4a36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:18:56.732070 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-70d72dbd-65f8-46ed-a898-b37342ed4a36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:18:56.733764 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1272/2498] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:18:56 2022] DELETE /identity/v3/projects/966f22e2bdf44625a350f3647b281d70 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:18:56.739219 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ff82f-49dc-495a-a391-6ddf8b1c4891 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:18:56.739637 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d20ff82f-49dc-495a-a391-6ddf8b1c4891 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:18:56.752649 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20ff82f-49dc-495a-a391-6ddf8b1c4891 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:18:56.810921 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb1997-87e7-458d-bc0d-cb8889949101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:18:56.811831 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb1997-87e7-458d-bc0d-cb8889949101 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:56.812118 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb1997-87e7-458d-bc0d-cb8889949101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:56.812397 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb1997-87e7-458d-bc0d-cb8889949101 admin admin] PATH_INFO: `/v3/users/8206715fa699482c9688bf24028b4001` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:56.813582 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bb1997-87e7-458d-bc0d-cb8889949101 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8206715fa699482c9688bf24028b4001)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:18:56.815511 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bb1997-87e7-458d-bc0d-cb8889949101 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:18:56.845718 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d72dbd-65f8-46ed-a898-b37342ed4a36 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:18:56.847134 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-70d72dbd-65f8-46ed-a898-b37342ed4a36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:18:56.920609 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d72dbd-65f8-46ed-a898-b37342ed4a36 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:18:56.926662 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f3bad-53b3-4ead-94ed-5cbc71507f96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:18:56.927317 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f3bad-53b3-4ead-94ed-5cbc71507f96 service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:56.927656 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f3bad-53b3-4ead-94ed-5cbc71507f96 service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:56.927920 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315f3bad-53b3-4ead-94ed-5cbc71507f96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:56.929808 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-315f3bad-53b3-4ead-94ed-5cbc71507f96 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:18:56.986403 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-89bb1997-87e7-458d-bc0d-cb8889949101 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8206715fa699482c9688bf24028b4001'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:18:56.992013 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315f3bad-53b3-4ead-94ed-5cbc71507f96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:18:56.993672 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315f3bad-53b3-4ead-94ed-5cbc71507f96 service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:18:56.999128 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-89bb1997-87e7-458d-bc0d-cb8889949101 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8206715fa699482c9688bf24028b4001'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:18:57.003866 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1273/2499] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:18:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:18:57.006165 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-89bb1997-87e7-458d-bc0d-cb8889949101 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8206715fa699482c9688bf24028b4001'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:18:57.006954 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-89bb1997-87e7-458d-bc0d-cb8889949101 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8206715fa699482c9688bf24028b4001'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:18:57.084633 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-89bb1997-87e7-458d-bc0d-cb8889949101 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8206715fa699482c9688bf24028b4001'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:18:57.091773 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1227/2500] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:18:56 2022] DELETE /identity/v3/users/8206715fa699482c9688bf24028b4001 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:18:57.354953 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f3bad-53b3-4ead-94ed-5cbc71507f96 service nova] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:18:57.356616 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-315f3bad-53b3-4ead-94ed-5cbc71507f96 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:18:57.373037 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315f3bad-53b3-4ead-94ed-5cbc71507f96 service nova] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:18:57.420107 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:18:57.436573 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7229e3-db1f-4c58-b290-b1466fb72c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:18:57.437569 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7229e3-db1f-4c58-b290-b1466fb72c68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:18:57.437853 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7229e3-db1f-4c58-b290-b1466fb72c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:18:57.438134 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7229e3-db1f-4c58-b290-b1466fb72c68 admin admin] PATH_INFO: `/v3/projects/71dd7e58775b460ebacf06d40f46fcc5` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:18:57.439330 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7229e3-db1f-4c58-b290-b1466fb72c68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71dd7e58775b460ebacf06d40f46fcc5)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:18:57.441381 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7229e3-db1f-4c58-b290-b1466fb72c68 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:18:57.554355 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1e7229e3-db1f-4c58-b290-b1466fb72c68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71dd7e58775b460ebacf06d40f46fcc5', 'request_id': 'req-1e7229e3-db1f-4c58-b290-b1466fb72c68'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:18:57.561955 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1e7229e3-db1f-4c58-b290-b1466fb72c68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71dd7e58775b460ebacf06d40f46fcc5', 'request_id': 'req-1e7229e3-db1f-4c58-b290-b1466fb72c68'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:18:57.567396 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1e7229e3-db1f-4c58-b290-b1466fb72c68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71dd7e58775b460ebacf06d40f46fcc5', 'request_id': 'req-1e7229e3-db1f-4c58-b290-b1466fb72c68'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:18:57.684095 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1e7229e3-db1f-4c58-b290-b1466fb72c68 admin admin] The token cache is being invalidate because project 71dd7e58775b460ebacf06d40f46fcc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:18:57.685246 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1e7229e3-db1f-4c58-b290-b1466fb72c68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:18:57.685551 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1e7229e3-db1f-4c58-b290-b1466fb72c68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:18:57.687112 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1274/2501] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:18:57 2022] DELETE /identity/v3/projects/71dd7e58775b460ebacf06d40f46fcc5 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:19:01.954921 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb1997-87e7-458d-bc0d-cb8889949101 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:01.956395 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-89bb1997-87e7-458d-bc0d-cb8889949101 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:01.971240 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb1997-87e7-458d-bc0d-cb8889949101 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:02.016041 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:19:02.032599 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff0514f-3a31-4e28-b400-9344806984f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:02.033525 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff0514f-3a31-4e28-b400-9344806984f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:02.033808 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff0514f-3a31-4e28-b400-9344806984f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:02.034084 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff0514f-3a31-4e28-b400-9344806984f3 admin admin] PATH_INFO: `/v3/users/bf57fc11325d46ac80b5b74ccf1d152d` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:02.035437 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff0514f-3a31-4e28-b400-9344806984f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf57fc11325d46ac80b5b74ccf1d152d)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:02.037526 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff0514f-3a31-4e28-b400-9344806984f3 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:02.183844 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-3ff0514f-3a31-4e28-b400-9344806984f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf57fc11325d46ac80b5b74ccf1d152d'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:02.196228 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-3ff0514f-3a31-4e28-b400-9344806984f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf57fc11325d46ac80b5b74ccf1d152d'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:02.203805 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-3ff0514f-3a31-4e28-b400-9344806984f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf57fc11325d46ac80b5b74ccf1d152d'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:02.204487 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-3ff0514f-3a31-4e28-b400-9344806984f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf57fc11325d46ac80b5b74ccf1d152d'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:02.284647 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-3ff0514f-3a31-4e28-b400-9344806984f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf57fc11325d46ac80b5b74ccf1d152d'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:02.290833 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1228/2502] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:19:01 2022] DELETE /identity/v3/users/bf57fc11325d46ac80b5b74ccf1d152d => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:19:02.295487 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35c4bd-3680-4629-8d47-8ae3b3fd8587 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:02.303927 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35c4bd-3680-4629-8d47-8ae3b3fd8587 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:02.303927 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe35c4bd-3680-4629-8d47-8ae3b3fd8587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:02.313956 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-fe35c4bd-3680-4629-8d47-8ae3b3fd8587 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:19:02.315123 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-fe35c4bd-3680-4629-8d47-8ae3b3fd8587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:02.375074 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1275/2503] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:19:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:19:02.384914 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff0514f-3a31-4e28-b400-9344806984f3 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:02.391814 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-3ff0514f-3a31-4e28-b400-9344806984f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:02.445573 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff0514f-3a31-4e28-b400-9344806984f3 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:02.449414 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f7d7b1-d6f7-4b68-85db-5b75fa4b9913 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:02.454743 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f7d7b1-d6f7-4b68-85db-5b75fa4b9913 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:02.454743 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f7d7b1-d6f7-4b68-85db-5b75fa4b9913 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:02.454743 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f7d7b1-d6f7-4b68-85db-5b75fa4b9913 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:02.461220 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f7d7b1-d6f7-4b68-85db-5b75fa4b9913 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:02.462662 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f7d7b1-d6f7-4b68-85db-5b75fa4b9913 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:02.473950 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1229/2504] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:19:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:19:03.000951 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35c4bd-3680-4629-8d47-8ae3b3fd8587 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:03.004509 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-fe35c4bd-3680-4629-8d47-8ae3b3fd8587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:03.025794 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe35c4bd-3680-4629-8d47-8ae3b3fd8587 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:03.033567 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf5fc45-54c1-4b7c-94e3-67df996f16b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:03.034445 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5fc45-54c1-4b7c-94e3-67df996f16b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:03.034725 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5fc45-54c1-4b7c-94e3-67df996f16b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:03.035159 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5fc45-54c1-4b7c-94e3-67df996f16b5 admin admin] PATH_INFO: `/v3/projects/a2e9151e312a4aa7a35553c029c99276` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:03.036435 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf5fc45-54c1-4b7c-94e3-67df996f16b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2e9151e312a4aa7a35553c029c99276)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:03.038379 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf5fc45-54c1-4b7c-94e3-67df996f16b5 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:03.159268 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-fbf5fc45-54c1-4b7c-94e3-67df996f16b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2e9151e312a4aa7a35553c029c99276', 'request_id': 'req-fbf5fc45-54c1-4b7c-94e3-67df996f16b5'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:03.165387 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-fbf5fc45-54c1-4b7c-94e3-67df996f16b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2e9151e312a4aa7a35553c029c99276', 'request_id': 'req-fbf5fc45-54c1-4b7c-94e3-67df996f16b5'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:03.171495 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-fbf5fc45-54c1-4b7c-94e3-67df996f16b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2e9151e312a4aa7a35553c029c99276', 'request_id': 'req-fbf5fc45-54c1-4b7c-94e3-67df996f16b5'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:03.175650 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:19:03.284756 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-fbf5fc45-54c1-4b7c-94e3-67df996f16b5 admin admin] The token cache is being invalidate because project a2e9151e312a4aa7a35553c029c99276 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:19:03.284967 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-fbf5fc45-54c1-4b7c-94e3-67df996f16b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:03.285629 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-fbf5fc45-54c1-4b7c-94e3-67df996f16b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:03.287419 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1276/2505] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:19:02 2022] DELETE /identity/v3/projects/a2e9151e312a4aa7a35553c029c99276 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:19:03.291562 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f7d7b1-d6f7-4b68-85db-5b75fa4b9913 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:03.302675 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f7d7b1-d6f7-4b68-85db-5b75fa4b9913 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:03.360288 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2636528-7689-4bca-9208-a429cf127d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:03.361196 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2636528-7689-4bca-9208-a429cf127d54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:03.361480 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2636528-7689-4bca-9208-a429cf127d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:03.361756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2636528-7689-4bca-9208-a429cf127d54 admin admin] PATH_INFO: `/v3/users/126aeecbc9be4a33a586d236655dd6e1` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:03.363024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2636528-7689-4bca-9208-a429cf127d54 admin admin] RBAC: Authorizing `identity:delete_user(user_id=126aeecbc9be4a33a586d236655dd6e1)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:03.365018 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2636528-7689-4bca-9208-a429cf127d54 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:03.370688 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:19:03.629423 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-c2636528-7689-4bca-9208-a429cf127d54 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '126aeecbc9be4a33a586d236655dd6e1'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:03.640868 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-c2636528-7689-4bca-9208-a429cf127d54 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '126aeecbc9be4a33a586d236655dd6e1'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:03.647582 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-c2636528-7689-4bca-9208-a429cf127d54 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '126aeecbc9be4a33a586d236655dd6e1'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:03.649273 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-c2636528-7689-4bca-9208-a429cf127d54 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '126aeecbc9be4a33a586d236655dd6e1'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:03.712829 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-c2636528-7689-4bca-9208-a429cf127d54 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '126aeecbc9be4a33a586d236655dd6e1'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:03.720041 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1230/2506] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:19:03 2022] DELETE /identity/v3/users/126aeecbc9be4a33a586d236655dd6e1 => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:19:04.143454 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf5fc45-54c1-4b7c-94e3-67df996f16b5 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:04.144962 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-fbf5fc45-54c1-4b7c-94e3-67df996f16b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:04.160948 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf5fc45-54c1-4b7c-94e3-67df996f16b5 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:04.219770 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c1ed5-ca17-4963-a08a-0742e8e90793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:04.220642 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c1ed5-ca17-4963-a08a-0742e8e90793 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:04.220929 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c1ed5-ca17-4963-a08a-0742e8e90793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:04.221210 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c1ed5-ca17-4963-a08a-0742e8e90793 admin admin] PATH_INFO: `/v3/projects/04f4d22d65ba42d194d937c01344c395` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:04.222435 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6c1ed5-ca17-4963-a08a-0742e8e90793 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04f4d22d65ba42d194d937c01344c395)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:04.224424 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6c1ed5-ca17-4963-a08a-0742e8e90793 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:04.364864 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8c6c1ed5-ca17-4963-a08a-0742e8e90793 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04f4d22d65ba42d194d937c01344c395', 'request_id': 'req-8c6c1ed5-ca17-4963-a08a-0742e8e90793'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:04.371127 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8c6c1ed5-ca17-4963-a08a-0742e8e90793 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04f4d22d65ba42d194d937c01344c395', 'request_id': 'req-8c6c1ed5-ca17-4963-a08a-0742e8e90793'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:04.377345 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8c6c1ed5-ca17-4963-a08a-0742e8e90793 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04f4d22d65ba42d194d937c01344c395', 'request_id': 'req-8c6c1ed5-ca17-4963-a08a-0742e8e90793'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:04.470618 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8c6c1ed5-ca17-4963-a08a-0742e8e90793 admin admin] The token cache is being invalidate because project 04f4d22d65ba42d194d937c01344c395 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:19:04.470929 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8c6c1ed5-ca17-4963-a08a-0742e8e90793 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:04.471597 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8c6c1ed5-ca17-4963-a08a-0742e8e90793 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:04.474165 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1277/2507] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:19:04 2022] DELETE /identity/v3/projects/04f4d22d65ba42d194d937c01344c395 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:19:04.479914 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255a564c-2ac5-4edb-bf5a-2224fd0af1e8 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:04.480416 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255a564c-2ac5-4edb-bf5a-2224fd0af1e8 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:04.480524 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255a564c-2ac5-4edb-bf5a-2224fd0af1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:04.499227 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-255a564c-2ac5-4edb-bf5a-2224fd0af1e8 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:19:04.500581 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-255a564c-2ac5-4edb-bf5a-2224fd0af1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:04.559817 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1231/2508] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:19:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:19:04.568363 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5bbae0-65cf-4adf-a58f-00c2152f59b2 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:04.568655 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5bbae0-65cf-4adf-a58f-00c2152f59b2 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:04.568931 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5bbae0-65cf-4adf-a58f-00c2152f59b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:04.587422 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-8b5bbae0-65cf-4adf-a58f-00c2152f59b2 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:19:04.588651 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-8b5bbae0-65cf-4adf-a58f-00c2152f59b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:04.596701 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1278/2509] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:19:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:19:04.600820 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255a564c-2ac5-4edb-bf5a-2224fd0af1e8 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:04.610609 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255a564c-2ac5-4edb-bf5a-2224fd0af1e8 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:04.615180 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415e4c2e-4ea5-4dde-aeec-e8f74b2ad074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:04.615889 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e4c2e-4ea5-4dde-aeec-e8f74b2ad074 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:04.616176 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e4c2e-4ea5-4dde-aeec-e8f74b2ad074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:04.616448 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e4c2e-4ea5-4dde-aeec-e8f74b2ad074 admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:04.617169 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415e4c2e-4ea5-4dde-aeec-e8f74b2ad074 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:19:04.617478 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415e4c2e-4ea5-4dde-aeec-e8f74b2ad074 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:04.619065 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415e4c2e-4ea5-4dde-aeec-e8f74b2ad074 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:04.631058 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1232/2510] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:19:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 17:19:04.635325 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5bbae0-65cf-4adf-a58f-00c2152f59b2 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:04.645155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5bbae0-65cf-4adf-a58f-00c2152f59b2 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:04.649881 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438c9982-5bb0-49d8-bb8c-8da59a90ff8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:04.650534 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c9982-5bb0-49d8-bb8c-8da59a90ff8b admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:04.650819 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c9982-5bb0-49d8-bb8c-8da59a90ff8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:04.651096 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c9982-5bb0-49d8-bb8c-8da59a90ff8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:04.651863 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438c9982-5bb0-49d8-bb8c-8da59a90ff8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:04.653789 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438c9982-5bb0-49d8-bb8c-8da59a90ff8b admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:04.738184 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1279/2511] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:19:04 2022] POST /identity/v3/projects => generated 371 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:19:04.742459 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415e4c2e-4ea5-4dde-aeec-e8f74b2ad074 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:04.753387 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415e4c2e-4ea5-4dde-aeec-e8f74b2ad074 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:04.813861 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017e2cd2-af2b-4773-9c15-9ee4cca5b15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:04.814570 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e2cd2-af2b-4773-9c15-9ee4cca5b15b admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:04.814853 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e2cd2-af2b-4773-9c15-9ee4cca5b15b admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:04.815130 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017e2cd2-af2b-4773-9c15-9ee4cca5b15b admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:04.815854 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017e2cd2-af2b-4773-9c15-9ee4cca5b15b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:04.817769 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017e2cd2-af2b-4773-9c15-9ee4cca5b15b admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:04.928452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1233/2512] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:19:04 2022] POST /identity/v3/users => generated 286 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:19:04.932858 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438c9982-5bb0-49d8-bb8c-8da59a90ff8b admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:04.944138 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438c9982-5bb0-49d8-bb8c-8da59a90ff8b admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:04.955741 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab017955-3622-42d8-b3f0-dfbbb33fe698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:04.955741 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab017955-3622-42d8-b3f0-dfbbb33fe698 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:04.955741 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab017955-3622-42d8-b3f0-dfbbb33fe698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:04.955741 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab017955-3622-42d8-b3f0-dfbbb33fe698 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:04.956173 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab017955-3622-42d8-b3f0-dfbbb33fe698 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:04.956173 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab017955-3622-42d8-b3f0-dfbbb33fe698 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:04.963441 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1280/2513] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:19:04.967522 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017e2cd2-af2b-4773-9c15-9ee4cca5b15b admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:04.972822 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:19:04.978158 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017e2cd2-af2b-4773-9c15-9ee4cca5b15b admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:04.983293 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8056b32-5c6d-405d-93cd-714d80314ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:04.984041 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8056b32-5c6d-405d-93cd-714d80314ad6 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:04.984295 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8056b32-5c6d-405d-93cd-714d80314ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:04.984537 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8056b32-5c6d-405d-93cd-714d80314ad6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:04.985313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8056b32-5c6d-405d-93cd-714d80314ad6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:04.986886 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8056b32-5c6d-405d-93cd-714d80314ad6 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:04.997532 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1234/2514] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:19:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:19:05.002140 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab017955-3622-42d8-b3f0-dfbbb33fe698 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:05.012148 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab017955-3622-42d8-b3f0-dfbbb33fe698 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:05.016798 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be664f-93d3-48be-b477-1a377707d19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:05.017672 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be664f-93d3-48be-b477-1a377707d19e admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:05.017963 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be664f-93d3-48be-b477-1a377707d19e admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:05.018240 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be664f-93d3-48be-b477-1a377707d19e admin admin] PATH_INFO: `/v3/projects/7964055dabbb439d9c1879467499adb2/users/6a14268ebb20478d8cbd039656eee7e4/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:05.033260 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36be664f-93d3-48be-b477-1a377707d19e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7964055dabbb439d9c1879467499adb2, user_id=6a14268ebb20478d8cbd039656eee7e4, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:05.035953 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36be664f-93d3-48be-b477-1a377707d19e admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:05.137073 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1281/2515] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:19:05 2022] PUT /identity/v3/projects/7964055dabbb439d9c1879467499adb2/users/6a14268ebb20478d8cbd039656eee7e4/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:19:05.148381 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ecae2-338a-4e61-9367-60dca4bdc16e None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:05.148674 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ecae2-338a-4e61-9367-60dca4bdc16e None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:05.148954 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015ecae2-338a-4e61-9367-60dca4bdc16e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:05.176520 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-015ecae2-338a-4e61-9367-60dca4bdc16e None None] MFA Rules not processed for user `6a14268ebb20478d8cbd039656eee7e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:19:05.177728 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-015ecae2-338a-4e61-9367-60dca4bdc16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:05.243643 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1235/2516] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:19:05 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:19:05.248111 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be664f-93d3-48be-b477-1a377707d19e admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:05.273001 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:19:05.309506 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be664f-93d3-48be-b477-1a377707d19e admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:05.314843 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad73a4-cbfa-4b50-b063-17e82ef82c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:05.316886 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad73a4-cbfa-4b50-b063-17e82ef82c32 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:05.316886 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad73a4-cbfa-4b50-b063-17e82ef82c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:05.316886 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad73a4-cbfa-4b50-b063-17e82ef82c32 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:05.316886 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ad73a4-cbfa-4b50-b063-17e82ef82c32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:05.318659 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ad73a4-cbfa-4b50-b063-17e82ef82c32 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:05.392507 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1282/2517] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:19:05 2022] POST /identity/v3/projects => generated 373 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:19:05.396613 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ecae2-338a-4e61-9367-60dca4bdc16e None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:05.408064 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015ecae2-338a-4e61-9367-60dca4bdc16e None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:05.464095 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bb563f-70fd-4460-9d1d-0f60d57e2999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:05.464766 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb563f-70fd-4460-9d1d-0f60d57e2999 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:05.465055 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb563f-70fd-4460-9d1d-0f60d57e2999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:05.465333 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb563f-70fd-4460-9d1d-0f60d57e2999 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:05.466029 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bb563f-70fd-4460-9d1d-0f60d57e2999 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:05.468630 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bb563f-70fd-4460-9d1d-0f60d57e2999 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:05.584887 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1236/2518] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:19:05 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:19:05.588759 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad73a4-cbfa-4b50-b063-17e82ef82c32 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:05.599524 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad73a4-cbfa-4b50-b063-17e82ef82c32 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:05.604129 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85194d53-7afc-4c85-a89f-b5ba4e84f855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:05.604783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85194d53-7afc-4c85-a89f-b5ba4e84f855 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:05.605065 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85194d53-7afc-4c85-a89f-b5ba4e84f855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:05.605340 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85194d53-7afc-4c85-a89f-b5ba4e84f855 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:05.606065 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85194d53-7afc-4c85-a89f-b5ba4e84f855 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:05.607669 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85194d53-7afc-4c85-a89f-b5ba4e84f855 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:05.617994 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1283/2519] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:19:05.621888 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bb563f-70fd-4460-9d1d-0f60d57e2999 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:05.631701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bb563f-70fd-4460-9d1d-0f60d57e2999 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:05.639531 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b141931-c0b0-49d8-b000-1514923ba19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:05.639531 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b141931-c0b0-49d8-b000-1514923ba19e admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:05.639531 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b141931-c0b0-49d8-b000-1514923ba19e admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:05.639531 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b141931-c0b0-49d8-b000-1514923ba19e admin admin] PATH_INFO: `/v3/projects/b256f942dba442f8af50d704d608b1e0/users/3f84e8660885433ca05521d941757e36/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:05.655893 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b141931-c0b0-49d8-b000-1514923ba19e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b256f942dba442f8af50d704d608b1e0, user_id=3f84e8660885433ca05521d941757e36, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:05.658660 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b141931-c0b0-49d8-b000-1514923ba19e admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:05.780826 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1237/2520] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:19:05 2022] PUT /identity/v3/projects/b256f942dba442f8af50d704d608b1e0/users/3f84e8660885433ca05521d941757e36/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:19:05.784444 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85194d53-7afc-4c85-a89f-b5ba4e84f855 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:05.794282 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85194d53-7afc-4c85-a89f-b5ba4e84f855 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:05.849913 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77441856-45a3-4a98-aaef-a7c49089abaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:05.850500 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77441856-45a3-4a98-aaef-a7c49089abaf admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:05.850779 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77441856-45a3-4a98-aaef-a7c49089abaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:05.851052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77441856-45a3-4a98-aaef-a7c49089abaf admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:05.851785 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77441856-45a3-4a98-aaef-a7c49089abaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:05.853350 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77441856-45a3-4a98-aaef-a7c49089abaf admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:05.866604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1284/2521] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:19:05 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:19:05.867749 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b141931-c0b0-49d8-b000-1514923ba19e admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:05.878075 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b141931-c0b0-49d8-b000-1514923ba19e admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:05.882929 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985647e3-a596-439f-a142-5c4771d61a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:05.884543 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985647e3-a596-439f-a142-5c4771d61a01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:05.884543 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985647e3-a596-439f-a142-5c4771d61a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:05.884543 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985647e3-a596-439f-a142-5c4771d61a01 admin admin] PATH_INFO: `/v3/projects/b256f942dba442f8af50d704d608b1e0/users/3f84e8660885433ca05521d941757e36/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:05.886346 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985647e3-a596-439f-a142-5c4771d61a01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b256f942dba442f8af50d704d608b1e0, user_id=3f84e8660885433ca05521d941757e36, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:05.889010 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985647e3-a596-439f-a142-5c4771d61a01 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:05.899502 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1238/2522] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:19:05 2022] PUT /identity/v3/projects/b256f942dba442f8af50d704d608b1e0/users/3f84e8660885433ca05521d941757e36/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:19:05.910501 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45730a76-0785-445f-9a73-064a2a1bae94 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:05.910796 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45730a76-0785-445f-9a73-064a2a1bae94 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:05.911080 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45730a76-0785-445f-9a73-064a2a1bae94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:05.939469 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-45730a76-0785-445f-9a73-064a2a1bae94 None None] MFA Rules not processed for user `3f84e8660885433ca05521d941757e36`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:19:05.940773 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-45730a76-0785-445f-9a73-064a2a1bae94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:06.008276 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1285/2523] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:19:05 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:19:06.019413 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985647e3-a596-439f-a142-5c4771d61a01 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:06.021112 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-985647e3-a596-439f-a142-5c4771d61a01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:06.056688 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:19:06.085349 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985647e3-a596-439f-a142-5c4771d61a01 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:06.089028 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e5d722-1dff-491e-a7e9-cfd361977e08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:06.089675 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5d722-1dff-491e-a7e9-cfd361977e08 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:06.089957 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5d722-1dff-491e-a7e9-cfd361977e08 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:06.090234 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5d722-1dff-491e-a7e9-cfd361977e08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:06.099579 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e5d722-1dff-491e-a7e9-cfd361977e08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:06.104951 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e5d722-1dff-491e-a7e9-cfd361977e08 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:06.110894 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1239/2524] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:19:06 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:19:07.699794 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45730a76-0785-445f-9a73-064a2a1bae94 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:07.719918 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:19:07.770853 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45730a76-0785-445f-9a73-064a2a1bae94 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:07.776481 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5df1e6-1aa5-4771-824d-5ee3b290196c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:07.787564 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5df1e6-1aa5-4771-824d-5ee3b290196c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:07.787564 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5df1e6-1aa5-4771-824d-5ee3b290196c admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:07.787564 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5df1e6-1aa5-4771-824d-5ee3b290196c admin admin] PATH_INFO: `/v3/users/6a14268ebb20478d8cbd039656eee7e4` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:07.787564 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5df1e6-1aa5-4771-824d-5ee3b290196c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a14268ebb20478d8cbd039656eee7e4)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:07.787564 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5df1e6-1aa5-4771-824d-5ee3b290196c admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:07.994548 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2b5df1e6-1aa5-4771-824d-5ee3b290196c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a14268ebb20478d8cbd039656eee7e4'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:08.001619 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2b5df1e6-1aa5-4771-824d-5ee3b290196c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a14268ebb20478d8cbd039656eee7e4'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:08.002275 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2b5df1e6-1aa5-4771-824d-5ee3b290196c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a14268ebb20478d8cbd039656eee7e4'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:08.014796 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2b5df1e6-1aa5-4771-824d-5ee3b290196c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a14268ebb20478d8cbd039656eee7e4'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:08.118458 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-2b5df1e6-1aa5-4771-824d-5ee3b290196c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a14268ebb20478d8cbd039656eee7e4'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:08.124748 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1286/2525] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:19:07 2022] DELETE /identity/v3/users/6a14268ebb20478d8cbd039656eee7e4 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:19:08.129745 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642b49f-6a24-4af4-8646-dd059dc7f001 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:08.130959 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642b49f-6a24-4af4-8646-dd059dc7f001 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:08.130959 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c642b49f-6a24-4af4-8646-dd059dc7f001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:08.148150 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-c642b49f-6a24-4af4-8646-dd059dc7f001 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:19:08.149301 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c642b49f-6a24-4af4-8646-dd059dc7f001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:08.210497 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1240/2526] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:19:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:19:08.220694 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5df1e6-1aa5-4771-824d-5ee3b290196c admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:08.222114 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-2b5df1e6-1aa5-4771-824d-5ee3b290196c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:08.278728 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5df1e6-1aa5-4771-824d-5ee3b290196c admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:08.283053 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2955db-23cc-4e89-abb0-4d6125d0ef2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:08.283696 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2955db-23cc-4e89-abb0-4d6125d0ef2a service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:08.284009 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2955db-23cc-4e89-abb0-4d6125d0ef2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:08.284282 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2955db-23cc-4e89-abb0-4d6125d0ef2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:08.293144 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2955db-23cc-4e89-abb0-4d6125d0ef2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:08.297506 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2955db-23cc-4e89-abb0-4d6125d0ef2a service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:08.304377 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1287/2527] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:19:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 17:19:08.709715 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c642b49f-6a24-4af4-8646-dd059dc7f001 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:08.711536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c642b49f-6a24-4af4-8646-dd059dc7f001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:08.733774 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c642b49f-6a24-4af4-8646-dd059dc7f001 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:08.738797 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c757499-f026-4861-aeb5-16616e4937d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:08.740210 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c757499-f026-4861-aeb5-16616e4937d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:08.740652 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c757499-f026-4861-aeb5-16616e4937d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:08.741088 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c757499-f026-4861-aeb5-16616e4937d1 admin admin] PATH_INFO: `/v3/projects/7964055dabbb439d9c1879467499adb2` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:08.742852 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c757499-f026-4861-aeb5-16616e4937d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7964055dabbb439d9c1879467499adb2)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:08.744870 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c757499-f026-4861-aeb5-16616e4937d1 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:08.890736 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-1c757499-f026-4861-aeb5-16616e4937d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7964055dabbb439d9c1879467499adb2', 'request_id': 'req-1c757499-f026-4861-aeb5-16616e4937d1'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:08.897762 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-1c757499-f026-4861-aeb5-16616e4937d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7964055dabbb439d9c1879467499adb2', 'request_id': 'req-1c757499-f026-4861-aeb5-16616e4937d1'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:08.973720 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-1c757499-f026-4861-aeb5-16616e4937d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7964055dabbb439d9c1879467499adb2', 'request_id': 'req-1c757499-f026-4861-aeb5-16616e4937d1'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:08.984981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-1c757499-f026-4861-aeb5-16616e4937d1 admin admin] The token cache is being invalidate because project 7964055dabbb439d9c1879467499adb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:19:08.985267 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-1c757499-f026-4861-aeb5-16616e4937d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:08.985897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-1c757499-f026-4861-aeb5-16616e4937d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:08.987417 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1241/2528] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:19:08 2022] DELETE /identity/v3/projects/7964055dabbb439d9c1879467499adb2 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:19:08.991394 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2955db-23cc-4e89-abb0-4d6125d0ef2a service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:09.001848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2955db-23cc-4e89-abb0-4d6125d0ef2a service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:09.040487 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:19:09.058420 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e857a9-14e6-4bbb-984e-921070314b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:09.059218 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e857a9-14e6-4bbb-984e-921070314b62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:09.059947 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e857a9-14e6-4bbb-984e-921070314b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:09.060233 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e857a9-14e6-4bbb-984e-921070314b62 admin admin] PATH_INFO: `/v3/users/3f84e8660885433ca05521d941757e36` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:09.061549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e857a9-14e6-4bbb-984e-921070314b62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f84e8660885433ca05521d941757e36)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:09.063883 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e857a9-14e6-4bbb-984e-921070314b62 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:09.269869 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-14e857a9-14e6-4bbb-984e-921070314b62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f84e8660885433ca05521d941757e36'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:09.276790 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-14e857a9-14e6-4bbb-984e-921070314b62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f84e8660885433ca05521d941757e36'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:09.277605 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-14e857a9-14e6-4bbb-984e-921070314b62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f84e8660885433ca05521d941757e36'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:09.290918 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-14e857a9-14e6-4bbb-984e-921070314b62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f84e8660885433ca05521d941757e36'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:09.377087 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-14e857a9-14e6-4bbb-984e-921070314b62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f84e8660885433ca05521d941757e36'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:09.385415 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1288/2529] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:19:08 2022] DELETE /identity/v3/users/3f84e8660885433ca05521d941757e36 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:19:09.937126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c757499-f026-4861-aeb5-16616e4937d1 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:09.941145 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-1c757499-f026-4861-aeb5-16616e4937d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:09.954288 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c757499-f026-4861-aeb5-16616e4937d1 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:10.010664 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5419aed7-db2d-43c0-bbd6-e160b03af490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:10.011533 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5419aed7-db2d-43c0-bbd6-e160b03af490 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:10.011863 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5419aed7-db2d-43c0-bbd6-e160b03af490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:10.012146 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5419aed7-db2d-43c0-bbd6-e160b03af490 admin admin] PATH_INFO: `/v3/projects/b256f942dba442f8af50d704d608b1e0` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:10.013349 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5419aed7-db2d-43c0-bbd6-e160b03af490 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b256f942dba442f8af50d704d608b1e0)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:10.015317 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5419aed7-db2d-43c0-bbd6-e160b03af490 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:10.017292 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:19:10.180958 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-5419aed7-db2d-43c0-bbd6-e160b03af490 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b256f942dba442f8af50d704d608b1e0', 'request_id': 'req-5419aed7-db2d-43c0-bbd6-e160b03af490'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:10.186972 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-5419aed7-db2d-43c0-bbd6-e160b03af490 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b256f942dba442f8af50d704d608b1e0', 'request_id': 'req-5419aed7-db2d-43c0-bbd6-e160b03af490'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:10.284544 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-5419aed7-db2d-43c0-bbd6-e160b03af490 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b256f942dba442f8af50d704d608b1e0', 'request_id': 'req-5419aed7-db2d-43c0-bbd6-e160b03af490'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:10.295395 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-5419aed7-db2d-43c0-bbd6-e160b03af490 admin admin] The token cache is being invalidate because project b256f942dba442f8af50d704d608b1e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:19:10.295659 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-5419aed7-db2d-43c0-bbd6-e160b03af490 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:10.296308 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-5419aed7-db2d-43c0-bbd6-e160b03af490 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:10.297848 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1242/2530] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:19:09 2022] DELETE /identity/v3/projects/b256f942dba442f8af50d704d608b1e0 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:19:10.303888 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9c35b-2ae3-4f34-980a-5eda7df8b047 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:10.304372 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9c35b-2ae3-4f34-980a-5eda7df8b047 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:10.304468 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c9c35b-2ae3-4f34-980a-5eda7df8b047 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:10.323113 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-07c9c35b-2ae3-4f34-980a-5eda7df8b047 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:19:10.324400 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-07c9c35b-2ae3-4f34-980a-5eda7df8b047 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:10.383266 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1289/2531] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:19:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:19:10.392864 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4cdf4-36b8-4847-8a88-2b4858927a07 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:10.393160 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4cdf4-36b8-4847-8a88-2b4858927a07 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:10.393443 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d4cdf4-36b8-4847-8a88-2b4858927a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:10.411909 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-a6d4cdf4-36b8-4847-8a88-2b4858927a07 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:19:10.413121 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-a6d4cdf4-36b8-4847-8a88-2b4858927a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:10.421858 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1243/2532] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:19:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:19:10.426358 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c9c35b-2ae3-4f34-980a-5eda7df8b047 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:10.436300 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c9c35b-2ae3-4f34-980a-5eda7df8b047 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:10.441478 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e381492-a333-4ae6-8398-02cc2009e623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:10.443225 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e381492-a333-4ae6-8398-02cc2009e623 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:10.446181 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e381492-a333-4ae6-8398-02cc2009e623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:10.446181 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e381492-a333-4ae6-8398-02cc2009e623 admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:10.446181 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e381492-a333-4ae6-8398-02cc2009e623 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:19:10.446181 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e381492-a333-4ae6-8398-02cc2009e623 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:10.447690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e381492-a333-4ae6-8398-02cc2009e623 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:10.461116 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1290/2533] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:19:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:19:10.465678 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d4cdf4-36b8-4847-8a88-2b4858927a07 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:10.475186 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d4cdf4-36b8-4847-8a88-2b4858927a07 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:10.479913 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a79a63-6a38-4b29-bd19-b37011f74622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:10.480713 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a79a63-6a38-4b29-bd19-b37011f74622 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:10.481037 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a79a63-6a38-4b29-bd19-b37011f74622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:10.481313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a79a63-6a38-4b29-bd19-b37011f74622 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:10.482042 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a79a63-6a38-4b29-bd19-b37011f74622 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:10.484050 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a79a63-6a38-4b29-bd19-b37011f74622 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:10.575203 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1244/2534] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:19:10 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:19:10.580105 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e381492-a333-4ae6-8398-02cc2009e623 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:10.590445 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e381492-a333-4ae6-8398-02cc2009e623 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:10.635796 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:19:10.648606 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3697821-016b-494a-abc5-266356fb99bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:10.649275 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3697821-016b-494a-abc5-266356fb99bc admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:10.649563 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3697821-016b-494a-abc5-266356fb99bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:10.649842 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3697821-016b-494a-abc5-266356fb99bc admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:10.650584 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3697821-016b-494a-abc5-266356fb99bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:10.652565 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3697821-016b-494a-abc5-266356fb99bc admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:10.747858 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1291/2535] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:19:10 2022] POST /identity/v3/users => generated 293 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:19:10.752567 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a79a63-6a38-4b29-bd19-b37011f74622 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:10.762870 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a79a63-6a38-4b29-bd19-b37011f74622 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:10.767648 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebab4dff-8f01-4889-aaa9-b4754bc116a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:10.768514 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab4dff-8f01-4889-aaa9-b4754bc116a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:10.769244 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab4dff-8f01-4889-aaa9-b4754bc116a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:10.769497 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab4dff-8f01-4889-aaa9-b4754bc116a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:10.770232 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebab4dff-8f01-4889-aaa9-b4754bc116a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:10.771859 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebab4dff-8f01-4889-aaa9-b4754bc116a4 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:10.782823 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1245/2536] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:19:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:19:10.786976 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3697821-016b-494a-abc5-266356fb99bc admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:10.798528 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3697821-016b-494a-abc5-266356fb99bc admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:10.803546 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c27ad3-aa20-43d9-bf17-43689e5086a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:10.804263 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c27ad3-aa20-43d9-bf17-43689e5086a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:10.804560 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c27ad3-aa20-43d9-bf17-43689e5086a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:10.804840 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c27ad3-aa20-43d9-bf17-43689e5086a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:10.805602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c27ad3-aa20-43d9-bf17-43689e5086a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:10.807192 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c27ad3-aa20-43d9-bf17-43689e5086a9 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:10.818165 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1292/2537] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:19:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:19:10.822181 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebab4dff-8f01-4889-aaa9-b4754bc116a4 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:10.832604 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebab4dff-8f01-4889-aaa9-b4754bc116a4 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:10.837149 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95beb9-4d28-419d-9639-aee2ae2cb60f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:10.838018 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95beb9-4d28-419d-9639-aee2ae2cb60f admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:10.838303 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95beb9-4d28-419d-9639-aee2ae2cb60f admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:10.838582 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95beb9-4d28-419d-9639-aee2ae2cb60f admin admin] PATH_INFO: `/v3/projects/0b7709e55e584c0899f9e9ff2a24ce93/users/d47d39221c9a4a2da9cae3d4326d1206/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:10.853307 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd95beb9-4d28-419d-9639-aee2ae2cb60f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b7709e55e584c0899f9e9ff2a24ce93, user_id=d47d39221c9a4a2da9cae3d4326d1206, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:10.856100 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd95beb9-4d28-419d-9639-aee2ae2cb60f admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:10.941705 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1246/2538] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:19:10 2022] PUT /identity/v3/projects/0b7709e55e584c0899f9e9ff2a24ce93/users/d47d39221c9a4a2da9cae3d4326d1206/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:19:10.952138 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026fb461-b88e-4fe4-837d-a0dc49c0b200 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:10.952373 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026fb461-b88e-4fe4-837d-a0dc49c0b200 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:10.952658 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026fb461-b88e-4fe4-837d-a0dc49c0b200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:10.980674 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-026fb461-b88e-4fe4-837d-a0dc49c0b200 None None] MFA Rules not processed for user `d47d39221c9a4a2da9cae3d4326d1206`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:19:10.981872 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-026fb461-b88e-4fe4-837d-a0dc49c0b200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:11.047413 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1293/2539] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:19:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:19:11.052238 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95beb9-4d28-419d-9639-aee2ae2cb60f admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:11.128710 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95beb9-4d28-419d-9639-aee2ae2cb60f admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:11.133374 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7328b2d2-e2a9-4faa-b777-be4320f82302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:11.134037 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7328b2d2-e2a9-4faa-b777-be4320f82302 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:11.134324 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7328b2d2-e2a9-4faa-b777-be4320f82302 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:11.134612 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7328b2d2-e2a9-4faa-b777-be4320f82302 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:11.135329 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7328b2d2-e2a9-4faa-b777-be4320f82302 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:11.137252 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7328b2d2-e2a9-4faa-b777-be4320f82302 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:11.289324 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1247/2540] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:19:11 2022] POST /identity/v3/projects => generated 385 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:19:11.293307 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026fb461-b88e-4fe4-837d-a0dc49c0b200 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:11.303710 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026fb461-b88e-4fe4-837d-a0dc49c0b200 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:11.358454 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3172e52e-b3c6-425a-8d9d-9559d095b593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:11.359626 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172e52e-b3c6-425a-8d9d-9559d095b593 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:11.359975 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172e52e-b3c6-425a-8d9d-9559d095b593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:11.360253 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172e52e-b3c6-425a-8d9d-9559d095b593 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:11.361009 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3172e52e-b3c6-425a-8d9d-9559d095b593 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:11.362921 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3172e52e-b3c6-425a-8d9d-9559d095b593 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:11.465558 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1294/2541] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:19:11 2022] POST /identity/v3/users => generated 299 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:19:11.469246 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7328b2d2-e2a9-4faa-b777-be4320f82302 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:11.479887 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7328b2d2-e2a9-4faa-b777-be4320f82302 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:11.485091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd9b61-2530-4264-929f-810b5ca4c60b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:11.487501 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd9b61-2530-4264-929f-810b5ca4c60b admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:11.487501 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd9b61-2530-4264-929f-810b5ca4c60b admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:11.487501 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdd9b61-2530-4264-929f-810b5ca4c60b admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:11.487501 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdd9b61-2530-4264-929f-810b5ca4c60b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:11.488796 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcdd9b61-2530-4264-929f-810b5ca4c60b admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:11.494074 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:19:11.501457 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1248/2542] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:19:11.505637 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3172e52e-b3c6-425a-8d9d-9559d095b593 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:11.515563 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3172e52e-b3c6-425a-8d9d-9559d095b593 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:11.520282 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c17ba6c-dbff-4b0a-ac58-5bffd60e91d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:11.521153 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c17ba6c-dbff-4b0a-ac58-5bffd60e91d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:11.521436 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c17ba6c-dbff-4b0a-ac58-5bffd60e91d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:11.521709 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c17ba6c-dbff-4b0a-ac58-5bffd60e91d0 admin admin] PATH_INFO: `/v3/projects/fe3e9026d70245cc983a229030bb7054/users/7874f7caa8444d68988e9406d6a26c2e/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:11.536412 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c17ba6c-dbff-4b0a-ac58-5bffd60e91d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe3e9026d70245cc983a229030bb7054, user_id=7874f7caa8444d68988e9406d6a26c2e, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:11.539046 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c17ba6c-dbff-4b0a-ac58-5bffd60e91d0 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:11.632581 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1295/2543] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:19:11 2022] PUT /identity/v3/projects/fe3e9026d70245cc983a229030bb7054/users/7874f7caa8444d68988e9406d6a26c2e/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:19:11.636355 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd9b61-2530-4264-929f-810b5ca4c60b admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:11.646490 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdd9b61-2530-4264-929f-810b5ca4c60b admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:11.702752 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dfb317-a4f2-42a1-87b0-383611187e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:11.703859 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfb317-a4f2-42a1-87b0-383611187e90 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:11.703859 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfb317-a4f2-42a1-87b0-383611187e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:11.704054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dfb317-a4f2-42a1-87b0-383611187e90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:11.704723 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dfb317-a4f2-42a1-87b0-383611187e90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:11.706323 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dfb317-a4f2-42a1-87b0-383611187e90 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:11.716764 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1249/2544] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:19:11 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:19:11.722227 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c17ba6c-dbff-4b0a-ac58-5bffd60e91d0 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:11.735975 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c17ba6c-dbff-4b0a-ac58-5bffd60e91d0 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:11.741148 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e5ea7-ca78-4d4f-9883-a74be244421e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:11.742197 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e5ea7-ca78-4d4f-9883-a74be244421e admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:11.742476 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e5ea7-ca78-4d4f-9883-a74be244421e admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:11.742829 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116e5ea7-ca78-4d4f-9883-a74be244421e admin admin] PATH_INFO: `/v3/projects/fe3e9026d70245cc983a229030bb7054/users/7874f7caa8444d68988e9406d6a26c2e/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:11.744860 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e5ea7-ca78-4d4f-9883-a74be244421e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe3e9026d70245cc983a229030bb7054, user_id=7874f7caa8444d68988e9406d6a26c2e, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:11.747670 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116e5ea7-ca78-4d4f-9883-a74be244421e admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:11.758425 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1296/2545] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:19:11 2022] PUT /identity/v3/projects/fe3e9026d70245cc983a229030bb7054/users/7874f7caa8444d68988e9406d6a26c2e/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:19:11.768817 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b041c9-ff4a-4de1-875f-de7787de6716 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:11.769116 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b041c9-ff4a-4de1-875f-de7787de6716 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:11.769402 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b041c9-ff4a-4de1-875f-de7787de6716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:11.798436 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-00b041c9-ff4a-4de1-875f-de7787de6716 None None] MFA Rules not processed for user `7874f7caa8444d68988e9406d6a26c2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:19:11.799664 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-00b041c9-ff4a-4de1-875f-de7787de6716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:11.880892 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1250/2546] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:19:11 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:19:11.891673 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e5ea7-ca78-4d4f-9883-a74be244421e admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:11.893523 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-116e5ea7-ca78-4d4f-9883-a74be244421e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:11.903271 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:19:11.950082 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116e5ea7-ca78-4d4f-9883-a74be244421e admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:11.953816 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f2a665-c7e4-40cc-8d47-4a6614193f49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:11.956918 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2a665-c7e4-40cc-8d47-4a6614193f49 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:11.956918 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2a665-c7e4-40cc-8d47-4a6614193f49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:11.956918 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2a665-c7e4-40cc-8d47-4a6614193f49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:12.009339 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f2a665-c7e4-40cc-8d47-4a6614193f49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:12.010764 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f2a665-c7e4-40cc-8d47-4a6614193f49 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:12.019419 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1297/2547] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:19:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:19:17.147558 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b041c9-ff4a-4de1-875f-de7787de6716 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:17.149044 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-00b041c9-ff4a-4de1-875f-de7787de6716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:17.161890 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b041c9-ff4a-4de1-875f-de7787de6716 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:17.165580 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eb9032-b142-4b7a-b571-48e73059864a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:17.166225 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb9032-b142-4b7a-b571-48e73059864a service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:17.166507 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb9032-b142-4b7a-b571-48e73059864a service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:17.166786 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb9032-b142-4b7a-b571-48e73059864a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:17.176595 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb9032-b142-4b7a-b571-48e73059864a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:17.178013 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eb9032-b142-4b7a-b571-48e73059864a service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:17.192155 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1251/2548] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:19:17 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:19:22.027191 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=45 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:19:23.018478 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f2a665-c7e4-40cc-8d47-4a6614193f49 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:23.085191 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f2a665-c7e4-40cc-8d47-4a6614193f49 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:23.090790 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d23f10e-e584-45df-ac66-003509b549d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:23.091673 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d23f10e-e584-45df-ac66-003509b549d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:23.091991 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d23f10e-e584-45df-ac66-003509b549d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:23.095380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d23f10e-e584-45df-ac66-003509b549d7 admin admin] PATH_INFO: `/v3/users/d47d39221c9a4a2da9cae3d4326d1206` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:23.095380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d23f10e-e584-45df-ac66-003509b549d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d47d39221c9a4a2da9cae3d4326d1206)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:23.095836 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d23f10e-e584-45df-ac66-003509b549d7 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:23.260223 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1d23f10e-e584-45df-ac66-003509b549d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd47d39221c9a4a2da9cae3d4326d1206'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:23.267276 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1d23f10e-e584-45df-ac66-003509b549d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd47d39221c9a4a2da9cae3d4326d1206'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:23.268096 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1d23f10e-e584-45df-ac66-003509b549d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd47d39221c9a4a2da9cae3d4326d1206'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:23.280578 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1d23f10e-e584-45df-ac66-003509b549d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd47d39221c9a4a2da9cae3d4326d1206'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:23.350403 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1d23f10e-e584-45df-ac66-003509b549d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd47d39221c9a4a2da9cae3d4326d1206'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:23.356761 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1298/2549] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:19:23 2022] DELETE /identity/v3/users/d47d39221c9a4a2da9cae3d4326d1206 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:19:23.361372 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8410aaa-576f-4b07-bb7b-280ed6ba24de None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:23.361894 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8410aaa-576f-4b07-bb7b-280ed6ba24de None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:23.361994 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8410aaa-576f-4b07-bb7b-280ed6ba24de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:23.379593 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-f8410aaa-576f-4b07-bb7b-280ed6ba24de None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:19:23.380806 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-f8410aaa-576f-4b07-bb7b-280ed6ba24de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:23.448492 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1252/2550] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:19:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:19:23.458899 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d23f10e-e584-45df-ac66-003509b549d7 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:23.460558 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-1d23f10e-e584-45df-ac66-003509b549d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:23.525482 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d23f10e-e584-45df-ac66-003509b549d7 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:23.529105 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0260a16-124e-4e6c-8467-6a8d54a8af54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:23.529811 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0260a16-124e-4e6c-8467-6a8d54a8af54 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:23.530366 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0260a16-124e-4e6c-8467-6a8d54a8af54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:23.530647 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0260a16-124e-4e6c-8467-6a8d54a8af54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:23.540777 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0260a16-124e-4e6c-8467-6a8d54a8af54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:23.542533 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0260a16-124e-4e6c-8467-6a8d54a8af54 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:23.552429 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1299/2551] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:19:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:19:23.933572 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f1b3b-f1a0-42c4-b567-580c04739ef5 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:23.933864 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f1b3b-f1a0-42c4-b567-580c04739ef5 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:23.934146 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f1b3b-f1a0-42c4-b567-580c04739ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:23.953685 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-1a3f1b3b-f1a0-42c4-b567-580c04739ef5 None None] MFA Rules not processed for user `1905e6f8c3e24e3cbfefce2827baa370`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:19:23.955613 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-1a3f1b3b-f1a0-42c4-b567-580c04739ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:23.960041 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:19:24.011648 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1253/2552] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 6 17:19:23 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:19:24.030601 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0260a16-124e-4e6c-8467-6a8d54a8af54 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:24.032356 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f0260a16-124e-4e6c-8467-6a8d54a8af54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:24.040772 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f1b3b-f1a0-42c4-b567-580c04739ef5 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:24.042698 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-1a3f1b3b-f1a0-42c4-b567-580c04739ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:24.060322 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f1b3b-f1a0-42c4-b567-580c04739ef5 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:24.065640 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:19:24.068654 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd9dfb1-7d9a-44db-a426-0a1a03866d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:24.069824 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd9dfb1-7d9a-44db-a426-0a1a03866d40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:24.070233 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd9dfb1-7d9a-44db-a426-0a1a03866d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:24.070525 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd9dfb1-7d9a-44db-a426-0a1a03866d40 admin admin] PATH_INFO: `/v3/projects/0b7709e55e584c0899f9e9ff2a24ce93` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:24.071937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd9dfb1-7d9a-44db-a426-0a1a03866d40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b7709e55e584c0899f9e9ff2a24ce93)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:24.076985 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd9dfb1-7d9a-44db-a426-0a1a03866d40 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:24.117742 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0260a16-124e-4e6c-8467-6a8d54a8af54 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:24.123172 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f87b35-53b7-4e29-b397-375b73d31cf6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75acacd30e9a4cf2ad58b072b49e3f05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:24.124513 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f87b35-53b7-4e29-b397-375b73d31cf6 service swift] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:24.124974 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f87b35-53b7-4e29-b397-375b73d31cf6 service swift] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:24.125409 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f87b35-53b7-4e29-b397-375b73d31cf6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:24.135299 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f87b35-53b7-4e29-b397-375b73d31cf6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:24.137047 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f87b35-53b7-4e29-b397-375b73d31cf6 service swift] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:24.146183 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1300/2553] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 6 17:19:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 17:19:24.208018 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-bbd9dfb1-7d9a-44db-a426-0a1a03866d40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b7709e55e584c0899f9e9ff2a24ce93', 'request_id': 'req-bbd9dfb1-7d9a-44db-a426-0a1a03866d40'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:24.214438 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-bbd9dfb1-7d9a-44db-a426-0a1a03866d40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b7709e55e584c0899f9e9ff2a24ce93', 'request_id': 'req-bbd9dfb1-7d9a-44db-a426-0a1a03866d40'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:24.295283 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-bbd9dfb1-7d9a-44db-a426-0a1a03866d40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b7709e55e584c0899f9e9ff2a24ce93', 'request_id': 'req-bbd9dfb1-7d9a-44db-a426-0a1a03866d40'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:24.325520 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-bbd9dfb1-7d9a-44db-a426-0a1a03866d40 admin admin] The token cache is being invalidate because project 0b7709e55e584c0899f9e9ff2a24ce93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:19:24.325520 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-bbd9dfb1-7d9a-44db-a426-0a1a03866d40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:24.325520 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-bbd9dfb1-7d9a-44db-a426-0a1a03866d40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:24.325520 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1254/2554] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:19:24 2022] DELETE /identity/v3/projects/0b7709e55e584c0899f9e9ff2a24ce93 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:19:24.326312 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f87b35-53b7-4e29-b397-375b73d31cf6 service swift] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:24.355658 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f87b35-53b7-4e29-b397-375b73d31cf6 service swift] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:24.454912 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec6538-e807-4be7-bada-db7407c05345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:24.455899 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec6538-e807-4be7-bada-db7407c05345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:24.456190 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec6538-e807-4be7-bada-db7407c05345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:24.456471 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec6538-e807-4be7-bada-db7407c05345 admin admin] PATH_INFO: `/v3/users/7874f7caa8444d68988e9406d6a26c2e` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:24.458168 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec6538-e807-4be7-bada-db7407c05345 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7874f7caa8444d68988e9406d6a26c2e)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:24.460373 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ec6538-e807-4be7-bada-db7407c05345 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:24.697793 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-24ec6538-e807-4be7-bada-db7407c05345 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7874f7caa8444d68988e9406d6a26c2e'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:24.713257 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-24ec6538-e807-4be7-bada-db7407c05345 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7874f7caa8444d68988e9406d6a26c2e'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:24.714132 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-24ec6538-e807-4be7-bada-db7407c05345 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7874f7caa8444d68988e9406d6a26c2e'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:24.744442 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-24ec6538-e807-4be7-bada-db7407c05345 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7874f7caa8444d68988e9406d6a26c2e'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:24.822473 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-24ec6538-e807-4be7-bada-db7407c05345 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7874f7caa8444d68988e9406d6a26c2e'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:24.832623 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1301/2555] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:19:24 2022] DELETE /identity/v3/users/7874f7caa8444d68988e9406d6a26c2e => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:19:25.111754 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd9dfb1-7d9a-44db-a426-0a1a03866d40 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:25.114419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-bbd9dfb1-7d9a-44db-a426-0a1a03866d40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:25.301751 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd9dfb1-7d9a-44db-a426-0a1a03866d40 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:25.351062 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:19:25.409157 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd94e79-af68-439c-9f5e-9be22f54b5b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:25.409994 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd94e79-af68-439c-9f5e-9be22f54b5b3 service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:25.410294 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd94e79-af68-439c-9f5e-9be22f54b5b3 service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:25.410582 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd94e79-af68-439c-9f5e-9be22f54b5b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:25.414294 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-7bd94e79-af68-439c-9f5e-9be22f54b5b3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:25.432398 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd94e79-af68-439c-9f5e-9be22f54b5b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:25.434167 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd94e79-af68-439c-9f5e-9be22f54b5b3 service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:25.447491 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1255/2556] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:19:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:19:25.515713 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec6538-e807-4be7-bada-db7407c05345 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:25.517862 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-24ec6538-e807-4be7-bada-db7407c05345 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:25.594896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:19:25.640501 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec6538-e807-4be7-bada-db7407c05345 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:25.646857 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bef6908-5e1f-40f9-be46-d2a2813c4db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:25.648276 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef6908-5e1f-40f9-be46-d2a2813c4db3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:25.648903 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef6908-5e1f-40f9-be46-d2a2813c4db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:25.649459 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bef6908-5e1f-40f9-be46-d2a2813c4db3 admin admin] PATH_INFO: `/v3/projects/fe3e9026d70245cc983a229030bb7054` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:25.650961 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bef6908-5e1f-40f9-be46-d2a2813c4db3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe3e9026d70245cc983a229030bb7054)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:25.654626 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bef6908-5e1f-40f9-be46-d2a2813c4db3 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:25.840812 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-4bef6908-5e1f-40f9-be46-d2a2813c4db3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe3e9026d70245cc983a229030bb7054', 'request_id': 'req-4bef6908-5e1f-40f9-be46-d2a2813c4db3'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:25.850946 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-4bef6908-5e1f-40f9-be46-d2a2813c4db3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe3e9026d70245cc983a229030bb7054', 'request_id': 'req-4bef6908-5e1f-40f9-be46-d2a2813c4db3'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:25.857884 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-4bef6908-5e1f-40f9-be46-d2a2813c4db3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe3e9026d70245cc983a229030bb7054', 'request_id': 'req-4bef6908-5e1f-40f9-be46-d2a2813c4db3'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:25.938749 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-4bef6908-5e1f-40f9-be46-d2a2813c4db3 admin admin] The token cache is being invalidate because project fe3e9026d70245cc983a229030bb7054 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:19:25.938749 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-4bef6908-5e1f-40f9-be46-d2a2813c4db3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:25.939488 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-4bef6908-5e1f-40f9-be46-d2a2813c4db3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:25.941842 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1302/2557] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:19:25 2022] DELETE /identity/v3/projects/fe3e9026d70245cc983a229030bb7054 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:19:25.947592 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476d2db-05f9-44bf-995e-8529e99d4d66 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:25.947927 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476d2db-05f9-44bf-995e-8529e99d4d66 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:25.948212 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1476d2db-05f9-44bf-995e-8529e99d4d66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:25.997335 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-1476d2db-05f9-44bf-995e-8529e99d4d66 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:19:25.999856 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-1476d2db-05f9-44bf-995e-8529e99d4d66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:26.137838 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1256/2558] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:19:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:19:26.147515 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03e2e3-0713-45d8-8cc7-f52464234f4e None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:26.148284 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03e2e3-0713-45d8-8cc7-f52464234f4e None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:26.148844 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03e2e3-0713-45d8-8cc7-f52464234f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:26.183156 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-7f03e2e3-0713-45d8-8cc7-f52464234f4e None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:19:26.185227 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-7f03e2e3-0713-45d8-8cc7-f52464234f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:26.196210 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1303/2559] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:19:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:19:26.200951 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1476d2db-05f9-44bf-995e-8529e99d4d66 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:26.223180 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1476d2db-05f9-44bf-995e-8529e99d4d66 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:26.235760 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde61631-3ff3-4cc4-8056-77e3ee7354e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:26.238781 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde61631-3ff3-4cc4-8056-77e3ee7354e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:26.238781 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde61631-3ff3-4cc4-8056-77e3ee7354e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:26.238781 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde61631-3ff3-4cc4-8056-77e3ee7354e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:26.239120 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde61631-3ff3-4cc4-8056-77e3ee7354e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:19:26.239693 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde61631-3ff3-4cc4-8056-77e3ee7354e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:26.242928 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde61631-3ff3-4cc4-8056-77e3ee7354e8 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:26.261408 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1257/2560] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:19:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:19:26.274848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f03e2e3-0713-45d8-8cc7-f52464234f4e None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:26.276480 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f03e2e3-0713-45d8-8cc7-f52464234f4e None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:26.284328 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc4f288-7dc6-4428-9e3b-596e75751265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:26.285102 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc4f288-7dc6-4428-9e3b-596e75751265 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:26.285774 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc4f288-7dc6-4428-9e3b-596e75751265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:26.286380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc4f288-7dc6-4428-9e3b-596e75751265 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:26.287088 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc4f288-7dc6-4428-9e3b-596e75751265 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:26.289086 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc4f288-7dc6-4428-9e3b-596e75751265 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:26.434394 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1304/2561] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:19:26 2022] POST /identity/v3/projects => generated 383 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:19:26.441178 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde61631-3ff3-4cc4-8056-77e3ee7354e8 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:26.453532 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde61631-3ff3-4cc4-8056-77e3ee7354e8 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:26.546569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6344a2-122c-4d82-ae8e-3461ae989ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:26.547403 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6344a2-122c-4d82-ae8e-3461ae989ab0 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:26.547752 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6344a2-122c-4d82-ae8e-3461ae989ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:26.548191 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6344a2-122c-4d82-ae8e-3461ae989ab0 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:26.548934 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6344a2-122c-4d82-ae8e-3461ae989ab0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:26.550983 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6344a2-122c-4d82-ae8e-3461ae989ab0 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:26.644631 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1258/2562] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:19:26 2022] POST /identity/v3/users => generated 292 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:19:26.650443 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc4f288-7dc6-4428-9e3b-596e75751265 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:26.666233 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc4f288-7dc6-4428-9e3b-596e75751265 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:26.677410 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc00be3-24fd-433d-a69a-6816e935a7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:26.678153 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc00be3-24fd-433d-a69a-6816e935a7c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:26.678445 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc00be3-24fd-433d-a69a-6816e935a7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:26.678726 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc00be3-24fd-433d-a69a-6816e935a7c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:26.679862 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc00be3-24fd-433d-a69a-6816e935a7c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:26.681133 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc00be3-24fd-433d-a69a-6816e935a7c2 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:26.700172 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1305/2563] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:19:26 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:19:26.702299 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6344a2-122c-4d82-ae8e-3461ae989ab0 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:26.718319 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6344a2-122c-4d82-ae8e-3461ae989ab0 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:26.726032 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f05f1c-f725-4568-837a-fe81632ff230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:26.726032 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f05f1c-f725-4568-837a-fe81632ff230 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:26.726032 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f05f1c-f725-4568-837a-fe81632ff230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:26.726032 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f05f1c-f725-4568-837a-fe81632ff230 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:26.729063 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f05f1c-f725-4568-837a-fe81632ff230 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:26.729142 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f05f1c-f725-4568-837a-fe81632ff230 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:26.755891 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1259/2564] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:19:26 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:19:26.759550 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc00be3-24fd-433d-a69a-6816e935a7c2 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:26.774087 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc00be3-24fd-433d-a69a-6816e935a7c2 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:26.779388 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884bc002-443f-4d44-82f7-17105406dd44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:26.780614 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884bc002-443f-4d44-82f7-17105406dd44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:26.781137 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884bc002-443f-4d44-82f7-17105406dd44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:26.781420 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884bc002-443f-4d44-82f7-17105406dd44 admin admin] PATH_INFO: `/v3/projects/5d4c472a5e62449d9bc6a853be09f7eb/users/06e3faddaeb74eb4b600bbfcad30e462/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:26.806656 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884bc002-443f-4d44-82f7-17105406dd44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d4c472a5e62449d9bc6a853be09f7eb, user_id=06e3faddaeb74eb4b600bbfcad30e462, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:26.809970 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884bc002-443f-4d44-82f7-17105406dd44 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:26.888420 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1306/2565] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:19:26 2022] PUT /identity/v3/projects/5d4c472a5e62449d9bc6a853be09f7eb/users/06e3faddaeb74eb4b600bbfcad30e462/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:19:26.894599 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf26fa6-9719-49d7-8a3a-7e55ec30b083 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:26.894599 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf26fa6-9719-49d7-8a3a-7e55ec30b083 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:26.894852 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf26fa6-9719-49d7-8a3a-7e55ec30b083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:26.922041 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-6cf26fa6-9719-49d7-8a3a-7e55ec30b083 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:19:26.926569 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-6cf26fa6-9719-49d7-8a3a-7e55ec30b083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:26.958117 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:19:27.002132 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1260/2566] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:19:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:19:27.014128 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884bc002-443f-4d44-82f7-17105406dd44 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:27.015868 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-884bc002-443f-4d44-82f7-17105406dd44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:27.098504 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884bc002-443f-4d44-82f7-17105406dd44 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:27.108315 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b1853-4dc7-40e8-902c-d8fa9f5c8743 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:27.109048 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b1853-4dc7-40e8-902c-d8fa9f5c8743 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:27.109355 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b1853-4dc7-40e8-902c-d8fa9f5c8743 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:27.109614 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b1853-4dc7-40e8-902c-d8fa9f5c8743 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:27.120258 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05b1853-4dc7-40e8-902c-d8fa9f5c8743 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:27.121695 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05b1853-4dc7-40e8-902c-d8fa9f5c8743 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:27.142340 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:19:27.154544 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1307/2567] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:19:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:19:29.508291 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb54d69-8cfb-41f7-ac87-b7227065998f None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:29.508886 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb54d69-8cfb-41f7-ac87-b7227065998f None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:29.509222 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb54d69-8cfb-41f7-ac87-b7227065998f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:29.554199 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-beb54d69-8cfb-41f7-ac87-b7227065998f None None] MFA Rules not processed for user `06e3faddaeb74eb4b600bbfcad30e462`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:19:29.555470 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-beb54d69-8cfb-41f7-ac87-b7227065998f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:29.659784 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1261/2568] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:19:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:19:29.672113 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b1853-4dc7-40e8-902c-d8fa9f5c8743 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:29.674942 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b05b1853-4dc7-40e8-902c-d8fa9f5c8743 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:29.779746 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b1853-4dc7-40e8-902c-d8fa9f5c8743 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:29.784427 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d6f6a2-6a8e-42b2-93e4-30659400c688 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:29.788730 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6f6a2-6a8e-42b2-93e4-30659400c688 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:29.788730 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6f6a2-6a8e-42b2-93e4-30659400c688 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:29.788730 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6f6a2-6a8e-42b2-93e4-30659400c688 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:29.795499 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d6f6a2-6a8e-42b2-93e4-30659400c688 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:29.797455 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d6f6a2-6a8e-42b2-93e4-30659400c688 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:29.814154 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1308/2569] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:19:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 17:19:31.717684 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe69a1f-3b7a-4f92-bf56-879b538c43e0 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:31.718211 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe69a1f-3b7a-4f92-bf56-879b538c43e0 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:31.718453 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe69a1f-3b7a-4f92-bf56-879b538c43e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:31.742320 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-abe69a1f-3b7a-4f92-bf56-879b538c43e0 None None] MFA Rules not processed for user `06e3faddaeb74eb4b600bbfcad30e462`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:19:31.743587 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-abe69a1f-3b7a-4f92-bf56-879b538c43e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:31.750877 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1262/2570] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:19:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:19:31.958734 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d6f6a2-6a8e-42b2-93e4-30659400c688 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:31.960365 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-67d6f6a2-6a8e-42b2-93e4-30659400c688 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:31.981344 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d6f6a2-6a8e-42b2-93e4-30659400c688 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:31.993067 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81350f40-5552-49c1-a230-01140fb538f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:31.994101 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81350f40-5552-49c1-a230-01140fb538f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:31.994557 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81350f40-5552-49c1-a230-01140fb538f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:31.994988 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81350f40-5552-49c1-a230-01140fb538f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:31.997064 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-81350f40-5552-49c1-a230-01140fb538f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:32.108352 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81350f40-5552-49c1-a230-01140fb538f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:32.110967 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81350f40-5552-49c1-a230-01140fb538f5 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:32.141301 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1309/2571] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:19:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:19:32.351775 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe69a1f-3b7a-4f92-bf56-879b538c43e0 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:32.367721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe69a1f-3b7a-4f92-bf56-879b538c43e0 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:32.376437 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e895e4-b8bd-4376-ae8f-2a6b8479ed35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:32.377135 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e895e4-b8bd-4376-ae8f-2a6b8479ed35 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:32.379026 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e895e4-b8bd-4376-ae8f-2a6b8479ed35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:32.379324 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e895e4-b8bd-4376-ae8f-2a6b8479ed35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:32.382423 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-92e895e4-b8bd-4376-ae8f-2a6b8479ed35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:32.397956 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e895e4-b8bd-4376-ae8f-2a6b8479ed35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:32.399858 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e895e4-b8bd-4376-ae8f-2a6b8479ed35 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:32.412298 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1263/2572] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:19:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:19:32.996251 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81350f40-5552-49c1-a230-01140fb538f5 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:33.007629 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81350f40-5552-49c1-a230-01140fb538f5 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:33.015993 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258d06d5-71d0-43e8-8fb0-7778dc7fd6f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:33.016756 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d06d5-71d0-43e8-8fb0-7778dc7fd6f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:33.017104 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d06d5-71d0-43e8-8fb0-7778dc7fd6f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:33.037619 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258d06d5-71d0-43e8-8fb0-7778dc7fd6f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:33.040353 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-258d06d5-71d0-43e8-8fb0-7778dc7fd6f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:33.044307 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e895e4-b8bd-4376-ae8f-2a6b8479ed35 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:33.046107 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-92e895e4-b8bd-4376-ae8f-2a6b8479ed35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:33.060306 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258d06d5-71d0-43e8-8fb0-7778dc7fd6f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:33.065426 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258d06d5-71d0-43e8-8fb0-7778dc7fd6f9 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:33.087941 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1310/2573] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:19:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:19:33.193910 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e895e4-b8bd-4376-ae8f-2a6b8479ed35 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:33.199851 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fc0604-8bd7-432a-9ba6-357aed6e5314 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76ca877bee9041d28ad41e45ef3dd18c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:33.200798 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc0604-8bd7-432a-9ba6-357aed6e5314 service placement] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:33.201303 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc0604-8bd7-432a-9ba6-357aed6e5314 service placement] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:33.201795 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc0604-8bd7-432a-9ba6-357aed6e5314 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:33.203870 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d8fc0604-8bd7-432a-9ba6-357aed6e5314 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:33.222655 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fc0604-8bd7-432a-9ba6-357aed6e5314 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:33.224590 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fc0604-8bd7-432a-9ba6-357aed6e5314 service placement] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:33.236334 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1264/2574] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:19:33 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:19:39.588508 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258d06d5-71d0-43e8-8fb0-7778dc7fd6f9 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:39.590689 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-258d06d5-71d0-43e8-8fb0-7778dc7fd6f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:39.604482 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258d06d5-71d0-43e8-8fb0-7778dc7fd6f9 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:39.609419 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add6dbda-6001-45af-83d5-611df0231e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:39.611242 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add6dbda-6001-45af-83d5-611df0231e3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:39.611624 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add6dbda-6001-45af-83d5-611df0231e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:39.611933 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add6dbda-6001-45af-83d5-611df0231e3d admin admin] PATH_INFO: `/v3/users/7922db6ad0ce41868a68f5a834a3b8f3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:39.613596 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add6dbda-6001-45af-83d5-611df0231e3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7922db6ad0ce41868a68f5a834a3b8f3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:39.616109 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add6dbda-6001-45af-83d5-611df0231e3d admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:39.853740 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-add6dbda-6001-45af-83d5-611df0231e3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7922db6ad0ce41868a68f5a834a3b8f3'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:39.861051 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-add6dbda-6001-45af-83d5-611df0231e3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7922db6ad0ce41868a68f5a834a3b8f3'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:39.861816 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-add6dbda-6001-45af-83d5-611df0231e3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7922db6ad0ce41868a68f5a834a3b8f3'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:39.873880 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-add6dbda-6001-45af-83d5-611df0231e3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7922db6ad0ce41868a68f5a834a3b8f3'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:39.993078 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-add6dbda-6001-45af-83d5-611df0231e3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7922db6ad0ce41868a68f5a834a3b8f3'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:39.996879 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:19:40.001899 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fc0604-8bd7-432a-9ba6-357aed6e5314 service placement] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:40.005054 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1311/2575] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:19:39 2022] DELETE /identity/v3/users/7922db6ad0ce41868a68f5a834a3b8f3 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:19:40.006783 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fc0604-8bd7-432a-9ba6-357aed6e5314 service placement] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:40.042943 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:19:40.071434 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c16b29-0b70-44ec-b2b2-28c9037548ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:40.081783 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c16b29-0b70-44ec-b2b2-28c9037548ba service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:40.081783 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c16b29-0b70-44ec-b2b2-28c9037548ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:40.081783 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c16b29-0b70-44ec-b2b2-28c9037548ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:40.081783 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-a5c16b29-0b70-44ec-b2b2-28c9037548ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:40.138437 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c16b29-0b70-44ec-b2b2-28c9037548ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:40.140097 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c16b29-0b70-44ec-b2b2-28c9037548ba service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:40.150667 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1265/2576] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:19:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:19:40.266380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add6dbda-6001-45af-83d5-611df0231e3d admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:40.268512 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-add6dbda-6001-45af-83d5-611df0231e3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:40.292528 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add6dbda-6001-45af-83d5-611df0231e3d admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:40.298822 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b55806-e066-4c2f-83df-be252db8efd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:40.300059 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b55806-e066-4c2f-83df-be252db8efd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:40.302535 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b55806-e066-4c2f-83df-be252db8efd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:40.302535 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b55806-e066-4c2f-83df-be252db8efd7 admin admin] PATH_INFO: `/v3/projects/f3d0d78164dc48e5aa8dc393c9071f4a` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:40.302535 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b55806-e066-4c2f-83df-be252db8efd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3d0d78164dc48e5aa8dc393c9071f4a)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:40.303809 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b55806-e066-4c2f-83df-be252db8efd7 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:40.426172 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-69b55806-e066-4c2f-83df-be252db8efd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3d0d78164dc48e5aa8dc393c9071f4a', 'request_id': 'req-69b55806-e066-4c2f-83df-be252db8efd7'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:40.432462 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-69b55806-e066-4c2f-83df-be252db8efd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3d0d78164dc48e5aa8dc393c9071f4a', 'request_id': 'req-69b55806-e066-4c2f-83df-be252db8efd7'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:40.438206 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-69b55806-e066-4c2f-83df-be252db8efd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3d0d78164dc48e5aa8dc393c9071f4a', 'request_id': 'req-69b55806-e066-4c2f-83df-be252db8efd7'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:40.538692 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-69b55806-e066-4c2f-83df-be252db8efd7 admin admin] The token cache is being invalidate because project f3d0d78164dc48e5aa8dc393c9071f4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:19:40.538983 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-69b55806-e066-4c2f-83df-be252db8efd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:40.539559 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-69b55806-e066-4c2f-83df-be252db8efd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:40.541161 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1312/2577] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:19:40 2022] DELETE /identity/v3/projects/f3d0d78164dc48e5aa8dc393c9071f4a => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:19:40.546194 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dfe961-d5c5-4f40-92af-5413fff8b856 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:40.546482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dfe961-d5c5-4f40-92af-5413fff8b856 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:40.550627 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dfe961-d5c5-4f40-92af-5413fff8b856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:40.566447 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-56dfe961-d5c5-4f40-92af-5413fff8b856 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:19:40.567652 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-56dfe961-d5c5-4f40-92af-5413fff8b856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:40.626925 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1266/2578] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:19:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:19:40.636834 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17f619-f68d-43c0-a8b9-7802f80bb3b9 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:40.641116 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17f619-f68d-43c0-a8b9-7802f80bb3b9 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:40.641116 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17f619-f68d-43c0-a8b9-7802f80bb3b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:40.657273 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-9e17f619-f68d-43c0-a8b9-7802f80bb3b9 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:19:40.658757 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-9e17f619-f68d-43c0-a8b9-7802f80bb3b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:40.669270 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1313/2579] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:19:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:19:40.674136 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dfe961-d5c5-4f40-92af-5413fff8b856 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:40.684126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dfe961-d5c5-4f40-92af-5413fff8b856 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:40.690584 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727de81e-2a6e-4a76-8bb1-ba4fbacc252e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:40.691446 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727de81e-2a6e-4a76-8bb1-ba4fbacc252e admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:40.691942 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727de81e-2a6e-4a76-8bb1-ba4fbacc252e admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:40.692374 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727de81e-2a6e-4a76-8bb1-ba4fbacc252e admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:40.693268 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727de81e-2a6e-4a76-8bb1-ba4fbacc252e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:19:40.693734 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727de81e-2a6e-4a76-8bb1-ba4fbacc252e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:40.695505 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727de81e-2a6e-4a76-8bb1-ba4fbacc252e admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:40.711613 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1267/2580] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:19:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:19:40.716387 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e17f619-f68d-43c0-a8b9-7802f80bb3b9 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:40.730836 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e17f619-f68d-43c0-a8b9-7802f80bb3b9 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:40.748970 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b9c1f-42a8-472d-85ae-a1534e028f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:40.749817 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b9c1f-42a8-472d-85ae-a1534e028f99 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:40.752189 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b9c1f-42a8-472d-85ae-a1534e028f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:40.752588 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861b9c1f-42a8-472d-85ae-a1534e028f99 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:40.753506 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b9c1f-42a8-472d-85ae-a1534e028f99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:40.758110 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861b9c1f-42a8-472d-85ae-a1534e028f99 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:40.911169 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1314/2581] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:19:40 2022] POST /identity/v3/projects => generated 383 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:19:40.912430 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727de81e-2a6e-4a76-8bb1-ba4fbacc252e admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:40.923850 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727de81e-2a6e-4a76-8bb1-ba4fbacc252e admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:40.986717 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b313879-e928-42ad-942c-e09c063cac5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:40.987409 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b313879-e928-42ad-942c-e09c063cac5c admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:40.987695 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b313879-e928-42ad-942c-e09c063cac5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:40.988012 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b313879-e928-42ad-942c-e09c063cac5c admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:40.988720 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b313879-e928-42ad-942c-e09c063cac5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:40.990678 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b313879-e928-42ad-942c-e09c063cac5c admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:40.999938 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:19:41.130864 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1268/2582] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:19:40 2022] POST /identity/v3/users => generated 292 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:19:41.135168 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b9c1f-42a8-472d-85ae-a1534e028f99 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:41.158698 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861b9c1f-42a8-472d-85ae-a1534e028f99 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:41.169475 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f64a1-2565-4452-b78f-d94d97c6344b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:41.170171 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f64a1-2565-4452-b78f-d94d97c6344b admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:41.170457 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f64a1-2565-4452-b78f-d94d97c6344b admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:41.170737 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f64a1-2565-4452-b78f-d94d97c6344b admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:41.171473 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3f64a1-2565-4452-b78f-d94d97c6344b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:41.173158 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3f64a1-2565-4452-b78f-d94d97c6344b admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:41.185363 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1315/2583] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:19:41 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:19:41.190144 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b313879-e928-42ad-942c-e09c063cac5c admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:41.201522 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b313879-e928-42ad-942c-e09c063cac5c admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:41.207101 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca965f35-53e1-4b3a-a5f7-de0e54cd3c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:41.208114 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca965f35-53e1-4b3a-a5f7-de0e54cd3c3a admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:41.208890 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca965f35-53e1-4b3a-a5f7-de0e54cd3c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:41.208890 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca965f35-53e1-4b3a-a5f7-de0e54cd3c3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:41.209680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca965f35-53e1-4b3a-a5f7-de0e54cd3c3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:41.211358 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca965f35-53e1-4b3a-a5f7-de0e54cd3c3a admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:41.224463 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1269/2584] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:19:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:19:41.229571 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f64a1-2565-4452-b78f-d94d97c6344b admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:41.241731 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f64a1-2565-4452-b78f-d94d97c6344b admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:41.246977 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f523801-19fb-461a-be3e-3b934a594d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:41.248437 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f523801-19fb-461a-be3e-3b934a594d8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:41.248725 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f523801-19fb-461a-be3e-3b934a594d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:41.249005 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f523801-19fb-461a-be3e-3b934a594d8f admin admin] PATH_INFO: `/v3/projects/cb01cd40add5405490508d2d14d1902d/users/ae705e3500914acda9d7a123b4c36298/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:41.276292 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f523801-19fb-461a-be3e-3b934a594d8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb01cd40add5405490508d2d14d1902d, user_id=ae705e3500914acda9d7a123b4c36298, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:41.280411 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f523801-19fb-461a-be3e-3b934a594d8f admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:41.365298 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1316/2585] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:19:41 2022] PUT /identity/v3/projects/cb01cd40add5405490508d2d14d1902d/users/ae705e3500914acda9d7a123b4c36298/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:19:41.370412 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01ecb2-7c6b-44cb-b994-7283d195b9b7 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:41.370826 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01ecb2-7c6b-44cb-b994-7283d195b9b7 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:41.371372 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01ecb2-7c6b-44cb-b994-7283d195b9b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:41.394108 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-fa01ecb2-7c6b-44cb-b994-7283d195b9b7 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:19:41.397959 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-fa01ecb2-7c6b-44cb-b994-7283d195b9b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:41.481667 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1270/2586] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:19:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:19:41.489934 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f523801-19fb-461a-be3e-3b934a594d8f admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:41.491828 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-9f523801-19fb-461a-be3e-3b934a594d8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:41.606222 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f523801-19fb-461a-be3e-3b934a594d8f admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:41.610320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b1403-086c-4bcf-950f-8fc33e781d9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:41.610980 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b1403-086c-4bcf-950f-8fc33e781d9b service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:41.611268 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b1403-086c-4bcf-950f-8fc33e781d9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:41.611548 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b1403-086c-4bcf-950f-8fc33e781d9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:41.627995 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7b1403-086c-4bcf-950f-8fc33e781d9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:41.627995 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7b1403-086c-4bcf-950f-8fc33e781d9b service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:41.639067 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1317/2587] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:19:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:19:44.759260 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462cf64-b371-4480-bfab-ef8a206ce6c1 None None] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:44.760152 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462cf64-b371-4480-bfab-ef8a206ce6c1 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:44.760456 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462cf64-b371-4480-bfab-ef8a206ce6c1 None None] PATH_INFO: `/` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:44.762729 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1271/2588] 10.222.0.40 () {54 vars in 921 bytes} [Wed Jul 6 17:19:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 06 17:19:44.770186 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f41b94-4fac-4a7c-b3c4-c1ed665696cd None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:44.770734 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f41b94-4fac-4a7c-b3c4-c1ed665696cd None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:44.771596 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f41b94-4fac-4a7c-b3c4-c1ed665696cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:44.803430 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-80f41b94-4fac-4a7c-b3c4-c1ed665696cd None None] MFA Rules not processed for user `af26ef96dc9d47dc91549b736939268e`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:19:44.804981 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-80f41b94-4fac-4a7c-b3c4-c1ed665696cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:44.818322 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:19:44.979833 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1318/2589] 10.222.0.40 () {58 vars in 1018 bytes} [Wed Jul 6 17:19:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:19:45.001002 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d462cf64-b371-4480-bfab-ef8a206ce6c1 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:45.003179 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d462cf64-b371-4480-bfab-ef8a206ce6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:45.089643 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d462cf64-b371-4480-bfab-ef8a206ce6c1 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:45.095565 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f0feb7-42cd-4b41-abc6-5927483b2fbf service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '635a410f471c43109438d0c39874070b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:45.096228 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0feb7-42cd-4b41-abc6-5927483b2fbf service glance] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:45.096516 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0feb7-42cd-4b41-abc6-5927483b2fbf service glance] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:45.096796 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0feb7-42cd-4b41-abc6-5927483b2fbf service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:45.111968 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f0feb7-42cd-4b41-abc6-5927483b2fbf service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:45.115003 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f0feb7-42cd-4b41-abc6-5927483b2fbf service glance] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:45.130790 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1272/2590] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:19:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:19:50.240486 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5118a2-daad-4095-8da7-d8135eb6edc9 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:50.241471 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5118a2-daad-4095-8da7-d8135eb6edc9 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:50.242777 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5118a2-daad-4095-8da7-d8135eb6edc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:50.309599 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-6a5118a2-daad-4095-8da7-d8135eb6edc9 None None] MFA Rules not processed for user `1905e6f8c3e24e3cbfefce2827baa370`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:19:50.310971 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-6a5118a2-daad-4095-8da7-d8135eb6edc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:50.453863 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a0039-45e9-499e-a461-2bc511642c4d None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:50.454274 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a0039-45e9-499e-a461-2bc511642c4d None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:50.454436 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9a0039-45e9-499e-a461-2bc511642c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:50.459938 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1319/2591] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 6 17:19:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:19:50.476242 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5118a2-daad-4095-8da7-d8135eb6edc9 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:50.478646 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-6a5118a2-daad-4095-8da7-d8135eb6edc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:50.533373 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-4c9a0039-45e9-499e-a461-2bc511642c4d None None] MFA Rules not processed for user `ae705e3500914acda9d7a123b4c36298`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:19:50.535822 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4c9a0039-45e9-499e-a461-2bc511642c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:50.606249 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5118a2-daad-4095-8da7-d8135eb6edc9 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:50.613259 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac929d-2e8e-4258-b858-cf38c71b120b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75acacd30e9a4cf2ad58b072b49e3f05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:50.614237 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac929d-2e8e-4258-b858-cf38c71b120b service swift] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:50.619128 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac929d-2e8e-4258-b858-cf38c71b120b service swift] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:50.619128 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac929d-2e8e-4258-b858-cf38c71b120b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:50.628319 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ac929d-2e8e-4258-b858-cf38c71b120b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:50.630270 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ac929d-2e8e-4258-b858-cf38c71b120b service swift] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:50.640150 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1320/2592] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 6 17:19:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 06 17:19:50.662696 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1273/2593] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:19:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:19:50.671393 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac929d-2e8e-4258-b858-cf38c71b120b service swift] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:50.672998 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-43ac929d-2e8e-4258-b858-cf38c71b120b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:50.710220 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac929d-2e8e-4258-b858-cf38c71b120b service swift] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:50.742969 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334ed8b8-704f-44b7-a2f8-8faaacba3702 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:50.743704 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334ed8b8-704f-44b7-a2f8-8faaacba3702 service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:50.743975 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334ed8b8-704f-44b7-a2f8-8faaacba3702 service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:50.744236 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334ed8b8-704f-44b7-a2f8-8faaacba3702 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:50.768711 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334ed8b8-704f-44b7-a2f8-8faaacba3702 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:50.770415 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334ed8b8-704f-44b7-a2f8-8faaacba3702 service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:50.780898 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1321/2594] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:19:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:19:51.843332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a0039-45e9-499e-a461-2bc511642c4d None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:51.847533 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4c9a0039-45e9-499e-a461-2bc511642c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:51.878723 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9a0039-45e9-499e-a461-2bc511642c4d None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:51.890201 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fa8755-abf7-4c3d-940e-2e81a09a1fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:51.902322 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa8755-abf7-4c3d-940e-2e81a09a1fda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:51.902796 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa8755-abf7-4c3d-940e-2e81a09a1fda admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:51.903715 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa8755-abf7-4c3d-940e-2e81a09a1fda admin admin] PATH_INFO: `/v3/users/2601402e569e491a94cea7a312cfef86` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:51.909536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fa8755-abf7-4c3d-940e-2e81a09a1fda admin admin] RBAC: Authorizing `identity:delete_user(user_id=2601402e569e491a94cea7a312cfef86)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:51.912847 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fa8755-abf7-4c3d-940e-2e81a09a1fda admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:51.990920 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:19:52.296828 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-60fa8755-abf7-4c3d-940e-2e81a09a1fda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2601402e569e491a94cea7a312cfef86'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:52.332168 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-60fa8755-abf7-4c3d-940e-2e81a09a1fda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2601402e569e491a94cea7a312cfef86'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:52.344810 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-60fa8755-abf7-4c3d-940e-2e81a09a1fda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2601402e569e491a94cea7a312cfef86'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:52.346288 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-60fa8755-abf7-4c3d-940e-2e81a09a1fda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2601402e569e491a94cea7a312cfef86'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:52.492658 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-60fa8755-abf7-4c3d-940e-2e81a09a1fda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2601402e569e491a94cea7a312cfef86'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:52.509614 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1274/2595] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:19:51 2022] DELETE /identity/v3/users/2601402e569e491a94cea7a312cfef86 => generated 0 bytes in 666 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:19:53.003755 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334ed8b8-704f-44b7-a2f8-8faaacba3702 service nova] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:53.010146 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fa8755-abf7-4c3d-940e-2e81a09a1fda admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:53.013517 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-60fa8755-abf7-4c3d-940e-2e81a09a1fda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:53.023131 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334ed8b8-704f-44b7-a2f8-8faaacba3702 service nova] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:53.100904 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:19:53.199189 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da01854a-dc0e-4f57-a9d3-0aa0b39c3e45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:53.199414 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fa8755-abf7-4c3d-940e-2e81a09a1fda admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:53.199811 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01854a-dc0e-4f57-a9d3-0aa0b39c3e45 service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:53.200106 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01854a-dc0e-4f57-a9d3-0aa0b39c3e45 service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:53.200387 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da01854a-dc0e-4f57-a9d3-0aa0b39c3e45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:53.205381 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-da01854a-dc0e-4f57-a9d3-0aa0b39c3e45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:53.210222 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050fff16-c6f8-4f81-ac8f-0561f47918df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:53.212815 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fff16-c6f8-4f81-ac8f-0561f47918df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:53.214920 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fff16-c6f8-4f81-ac8f-0561f47918df admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:53.226942 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fff16-c6f8-4f81-ac8f-0561f47918df admin admin] PATH_INFO: `/v3/projects/9f85d0f3814648d18a627e1353493973` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:53.228795 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050fff16-c6f8-4f81-ac8f-0561f47918df admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f85d0f3814648d18a627e1353493973)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:53.232650 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050fff16-c6f8-4f81-ac8f-0561f47918df admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:53.336587 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da01854a-dc0e-4f57-a9d3-0aa0b39c3e45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:53.339618 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da01854a-dc0e-4f57-a9d3-0aa0b39c3e45 service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:53.379140 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1322/2596] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:19:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:19:53.419444 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-050fff16-c6f8-4f81-ac8f-0561f47918df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f85d0f3814648d18a627e1353493973', 'request_id': 'req-050fff16-c6f8-4f81-ac8f-0561f47918df'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:53.435625 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-050fff16-c6f8-4f81-ac8f-0561f47918df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f85d0f3814648d18a627e1353493973', 'request_id': 'req-050fff16-c6f8-4f81-ac8f-0561f47918df'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:53.583293 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-050fff16-c6f8-4f81-ac8f-0561f47918df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f85d0f3814648d18a627e1353493973', 'request_id': 'req-050fff16-c6f8-4f81-ac8f-0561f47918df'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:53.631632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-050fff16-c6f8-4f81-ac8f-0561f47918df admin admin] The token cache is being invalidate because project 9f85d0f3814648d18a627e1353493973 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:19:53.632293 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-050fff16-c6f8-4f81-ac8f-0561f47918df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:53.638809 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-050fff16-c6f8-4f81-ac8f-0561f47918df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:53.641007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1275/2597] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:19:53 2022] DELETE /identity/v3/projects/9f85d0f3814648d18a627e1353493973 => generated 0 bytes in 632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:19:53.646815 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da01854a-dc0e-4f57-a9d3-0aa0b39c3e45 service nova] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:53.666542 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da01854a-dc0e-4f57-a9d3-0aa0b39c3e45 service nova] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:53.856715 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cf44be-86cc-4b63-bd9f-344337f7af9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:53.857646 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf44be-86cc-4b63-bd9f-344337f7af9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:53.857938 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf44be-86cc-4b63-bd9f-344337f7af9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:53.858282 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cf44be-86cc-4b63-bd9f-344337f7af9e admin admin] PATH_INFO: `/v3/users/eb0e1e84f287480880a3150164e5a467` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:53.859647 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cf44be-86cc-4b63-bd9f-344337f7af9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb0e1e84f287480880a3150164e5a467)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:53.862663 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4cf44be-86cc-4b63-bd9f-344337f7af9e admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:54.178285 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-b4cf44be-86cc-4b63-bd9f-344337f7af9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb0e1e84f287480880a3150164e5a467'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:54.191214 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-b4cf44be-86cc-4b63-bd9f-344337f7af9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb0e1e84f287480880a3150164e5a467'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:54.192241 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-b4cf44be-86cc-4b63-bd9f-344337f7af9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb0e1e84f287480880a3150164e5a467'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:54.202575 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:19:54.211783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-b4cf44be-86cc-4b63-bd9f-344337f7af9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb0e1e84f287480880a3150164e5a467'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:54.316130 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-b4cf44be-86cc-4b63-bd9f-344337f7af9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb0e1e84f287480880a3150164e5a467'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:54.327690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1323/2598] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:19:53 2022] DELETE /identity/v3/users/eb0e1e84f287480880a3150164e5a467 => generated 0 bytes in 679 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:19:54.703241 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050fff16-c6f8-4f81-ac8f-0561f47918df admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:19:54.705335 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-050fff16-c6f8-4f81-ac8f-0561f47918df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:54.751659 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050fff16-c6f8-4f81-ac8f-0561f47918df admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:19:54.820042 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:19:54.979537 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a124b-59a1-44a9-915b-187e27a2cac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:19:54.980895 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a124b-59a1-44a9-915b-187e27a2cac6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:54.981343 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a124b-59a1-44a9-915b-187e27a2cac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:54.981809 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832a124b-59a1-44a9-915b-187e27a2cac6 admin admin] PATH_INFO: `/v3/projects/9f3fa70ae6ea49568bd152e70b5f7b86` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:55.012429 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832a124b-59a1-44a9-915b-187e27a2cac6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f3fa70ae6ea49568bd152e70b5f7b86)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:19:55.012429 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832a124b-59a1-44a9-915b-187e27a2cac6 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:19:55.227778 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-832a124b-59a1-44a9-915b-187e27a2cac6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f3fa70ae6ea49568bd152e70b5f7b86', 'request_id': 'req-832a124b-59a1-44a9-915b-187e27a2cac6'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:55.248421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-832a124b-59a1-44a9-915b-187e27a2cac6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f3fa70ae6ea49568bd152e70b5f7b86', 'request_id': 'req-832a124b-59a1-44a9-915b-187e27a2cac6'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:55.341728 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-832a124b-59a1-44a9-915b-187e27a2cac6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f3fa70ae6ea49568bd152e70b5f7b86', 'request_id': 'req-832a124b-59a1-44a9-915b-187e27a2cac6'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:55.366934 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-832a124b-59a1-44a9-915b-187e27a2cac6 admin admin] The token cache is being invalidate because project 9f3fa70ae6ea49568bd152e70b5f7b86 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:19:55.367477 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-832a124b-59a1-44a9-915b-187e27a2cac6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:55.370435 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-832a124b-59a1-44a9-915b-187e27a2cac6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:19:55.373242 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1276/2599] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:19:54 2022] DELETE /identity/v3/projects/9f3fa70ae6ea49568bd152e70b5f7b86 => generated 0 bytes in 670 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:19:55.580757 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f15b2-b4b6-4d6d-bde9-82152583762b None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:19:55.581292 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f15b2-b4b6-4d6d-bde9-82152583762b None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:19:55.581406 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16f15b2-b4b6-4d6d-bde9-82152583762b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:19:55.617703 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-a16f15b2-b4b6-4d6d-bde9-82152583762b None None] MFA Rules not processed for user `ae705e3500914acda9d7a123b4c36298`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:19:55.619924 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-a16f15b2-b4b6-4d6d-bde9-82152583762b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:19:55.772654 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1324/2600] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:19:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:20:01.804322 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a124b-59a1-44a9-915b-187e27a2cac6 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:20:01.805764 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-832a124b-59a1-44a9-915b-187e27a2cac6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:20:01.862933 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832a124b-59a1-44a9-915b-187e27a2cac6 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:20:01.866652 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6b2a8d-73f5-4b04-ad31-4a85e0d29b29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:20:01.867302 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b2a8d-73f5-4b04-ad31-4a85e0d29b29 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:20:01.867591 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b2a8d-73f5-4b04-ad31-4a85e0d29b29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:20:01.867910 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b2a8d-73f5-4b04-ad31-4a85e0d29b29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:20:01.869716 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-7f6b2a8d-73f5-4b04-ad31-4a85e0d29b29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:20:01.879939 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6b2a8d-73f5-4b04-ad31-4a85e0d29b29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:20:01.881665 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6b2a8d-73f5-4b04-ad31-4a85e0d29b29 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:20:01.889880 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1277/2601] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:20:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:20:05.739081 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=30,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:20:09.835639 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e8295-6a50-4d62-80e6-286e3b0c9550 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:20:09.836260 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e8295-6a50-4d62-80e6-286e3b0c9550 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:20:09.836361 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e8295-6a50-4d62-80e6-286e3b0c9550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:20:09.853395 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-e00e8295-6a50-4d62-80e6-286e3b0c9550 None None] MFA Rules not processed for user `1905e6f8c3e24e3cbfefce2827baa370`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:20:09.854566 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-e00e8295-6a50-4d62-80e6-286e3b0c9550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:20:09.908082 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1325/2602] 10.222.0.40 () {58 vars in 967 bytes} [Wed Jul 6 17:20:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:20:09.920234 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6b2a8d-73f5-4b04-ad31-4a85e0d29b29 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:20:09.921605 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-7f6b2a8d-73f5-4b04-ad31-4a85e0d29b29 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:20:09.973470 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:20:09.990803 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6b2a8d-73f5-4b04-ad31-4a85e0d29b29 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:20:09.995910 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db69dd0c-116b-4df7-ba97-621631ce23db service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75acacd30e9a4cf2ad58b072b49e3f05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:20:09.996591 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db69dd0c-116b-4df7-ba97-621631ce23db service swift] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:20:09.996872 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db69dd0c-116b-4df7-ba97-621631ce23db service swift] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:20:09.997241 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db69dd0c-116b-4df7-ba97-621631ce23db service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:20:10.005522 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db69dd0c-116b-4df7-ba97-621631ce23db service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:20:10.007110 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db69dd0c-116b-4df7-ba97-621631ce23db service swift] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:20:10.015157 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1278/2603] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 6 17:20:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 17:20:19.901451 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=25 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:20:20.054954 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=9 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:20:46.670631 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6500542-da66-4902-8b23-ba07ff12ee48 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:20:46.674832 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6500542-da66-4902-8b23-ba07ff12ee48 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:20:46.674832 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6500542-da66-4902-8b23-ba07ff12ee48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:20:46.689141 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-d6500542-da66-4902-8b23-ba07ff12ee48 None None] MFA Rules not processed for user `06e3faddaeb74eb4b600bbfcad30e462`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:20:46.690337 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d6500542-da66-4902-8b23-ba07ff12ee48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:20:46.748075 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1326/2604] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:20:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:20:56.741214 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=25 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:21:00.831989 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db69dd0c-116b-4df7-ba97-621631ce23db service swift] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:21:00.841461 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db69dd0c-116b-4df7-ba97-621631ce23db service swift] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:21:00.845169 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e1255-0a54-4e0b-a839-7261a4c9c838 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:21:00.845811 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e1255-0a54-4e0b-a839-7261a4c9c838 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:21:00.846095 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e1255-0a54-4e0b-a839-7261a4c9c838 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:21:00.846368 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e1255-0a54-4e0b-a839-7261a4c9c838 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:21:00.848168 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-034e1255-0a54-4e0b-a839-7261a4c9c838 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:21:00.910917 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034e1255-0a54-4e0b-a839-7261a4c9c838 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:21:00.912408 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034e1255-0a54-4e0b-a839-7261a4c9c838 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:21:00.922977 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1279/2605] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:21:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:21:04.762800 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6500542-da66-4902-8b23-ba07ff12ee48 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:21:04.765167 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d6500542-da66-4902-8b23-ba07ff12ee48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:21:04.780370 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6500542-da66-4902-8b23-ba07ff12ee48 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:21:04.785459 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287df46f-cab0-4711-92ed-5253a54730fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:21:04.786458 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287df46f-cab0-4711-92ed-5253a54730fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:21:04.786813 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287df46f-cab0-4711-92ed-5253a54730fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:21:04.787093 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287df46f-cab0-4711-92ed-5253a54730fa admin admin] PATH_INFO: `/v3/users/8b5fdc57573c421195439a0c8e95d7d4` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:21:04.788439 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287df46f-cab0-4711-92ed-5253a54730fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b5fdc57573c421195439a0c8e95d7d4)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:21:04.790887 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287df46f-cab0-4711-92ed-5253a54730fa admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:21:05.020753 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-287df46f-cab0-4711-92ed-5253a54730fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b5fdc57573c421195439a0c8e95d7d4'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:21:05.028695 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-287df46f-cab0-4711-92ed-5253a54730fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b5fdc57573c421195439a0c8e95d7d4'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:21:05.029422 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-287df46f-cab0-4711-92ed-5253a54730fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b5fdc57573c421195439a0c8e95d7d4'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:21:05.041880 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-287df46f-cab0-4711-92ed-5253a54730fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b5fdc57573c421195439a0c8e95d7d4'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:21:05.139902 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-287df46f-cab0-4711-92ed-5253a54730fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b5fdc57573c421195439a0c8e95d7d4'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:21:05.146077 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1327/2606] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:21:04 2022] DELETE /identity/v3/users/8b5fdc57573c421195439a0c8e95d7d4 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:21:05.422545 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e1255-0a54-4e0b-a839-7261a4c9c838 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:21:05.434429 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e1255-0a54-4e0b-a839-7261a4c9c838 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:21:05.498157 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57228fda-ba7e-4a33-9760-bf65db0544b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:21:05.499029 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57228fda-ba7e-4a33-9760-bf65db0544b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:21:05.499420 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57228fda-ba7e-4a33-9760-bf65db0544b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:21:05.499580 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57228fda-ba7e-4a33-9760-bf65db0544b6 admin admin] PATH_INFO: `/v3/projects/b1d0f8e711df49f58773fd76227ee878` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:21:05.500946 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57228fda-ba7e-4a33-9760-bf65db0544b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1d0f8e711df49f58773fd76227ee878)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:21:05.502849 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57228fda-ba7e-4a33-9760-bf65db0544b6 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:21:05.608777 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-57228fda-ba7e-4a33-9760-bf65db0544b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1d0f8e711df49f58773fd76227ee878', 'request_id': 'req-57228fda-ba7e-4a33-9760-bf65db0544b6'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:21:05.614921 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-57228fda-ba7e-4a33-9760-bf65db0544b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1d0f8e711df49f58773fd76227ee878', 'request_id': 'req-57228fda-ba7e-4a33-9760-bf65db0544b6'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:21:05.700543 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-57228fda-ba7e-4a33-9760-bf65db0544b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1d0f8e711df49f58773fd76227ee878', 'request_id': 'req-57228fda-ba7e-4a33-9760-bf65db0544b6'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:21:05.716285 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-57228fda-ba7e-4a33-9760-bf65db0544b6 admin admin] The token cache is being invalidate because project b1d0f8e711df49f58773fd76227ee878 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:21:05.716587 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-57228fda-ba7e-4a33-9760-bf65db0544b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:21:05.717252 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-57228fda-ba7e-4a33-9760-bf65db0544b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:21:05.718898 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1280/2607] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:21:05 2022] DELETE /identity/v3/projects/b1d0f8e711df49f58773fd76227ee878 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:21:05.724013 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738b9c4-1203-432f-86d9-c3c21c220612 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:21:05.724307 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738b9c4-1203-432f-86d9-c3c21c220612 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:21:05.724599 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4738b9c4-1203-432f-86d9-c3c21c220612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:21:05.744260 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-4738b9c4-1203-432f-86d9-c3c21c220612 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:21:05.745460 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-4738b9c4-1203-432f-86d9-c3c21c220612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:21:05.803070 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1328/2608] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:21:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:21:05.811311 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d72ca6-ba6b-4c52-b45b-63bb78c6cb67 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:21:05.811607 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d72ca6-ba6b-4c52-b45b-63bb78c6cb67 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:21:05.811932 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d72ca6-ba6b-4c52-b45b-63bb78c6cb67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:21:05.829890 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-d4d72ca6-ba6b-4c52-b45b-63bb78c6cb67 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:21:05.831100 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d4d72ca6-ba6b-4c52-b45b-63bb78c6cb67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:21:05.839511 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1281/2609] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:21:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:21:05.843566 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4738b9c4-1203-432f-86d9-c3c21c220612 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:21:05.853706 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4738b9c4-1203-432f-86d9-c3c21c220612 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:21:05.858229 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c58fbad-3f3c-4453-ac98-702ae25388a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:21:05.858872 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c58fbad-3f3c-4453-ac98-702ae25388a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:21:05.859155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c58fbad-3f3c-4453-ac98-702ae25388a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:21:05.859427 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c58fbad-3f3c-4453-ac98-702ae25388a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:21:05.860174 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c58fbad-3f3c-4453-ac98-702ae25388a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:21:05.860480 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c58fbad-3f3c-4453-ac98-702ae25388a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:21:05.862056 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c58fbad-3f3c-4453-ac98-702ae25388a4 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:21:05.873317 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1329/2610] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:21:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 17:21:05.877391 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d72ca6-ba6b-4c52-b45b-63bb78c6cb67 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:21:05.886287 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d72ca6-ba6b-4c52-b45b-63bb78c6cb67 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:21:05.890859 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b2d10-33d8-4bfc-8153-f03f439c1684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:21:05.891499 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b2d10-33d8-4bfc-8153-f03f439c1684 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:21:05.891811 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b2d10-33d8-4bfc-8153-f03f439c1684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:21:05.892091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418b2d10-33d8-4bfc-8153-f03f439c1684 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:21:05.892780 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418b2d10-33d8-4bfc-8153-f03f439c1684 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:21:05.894682 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418b2d10-33d8-4bfc-8153-f03f439c1684 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:21:05.908808 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:21:06.053486 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1282/2611] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:21:05 2022] POST /identity/v3/projects => generated 371 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:21:06.057611 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c58fbad-3f3c-4453-ac98-702ae25388a4 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:21:06.067423 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c58fbad-3f3c-4453-ac98-702ae25388a4 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:21:06.122050 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f921ef66-e316-47ba-a98b-e618bb828f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:21:06.122716 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921ef66-e316-47ba-a98b-e618bb828f60 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:21:06.123020 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921ef66-e316-47ba-a98b-e618bb828f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:21:06.123296 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f921ef66-e316-47ba-a98b-e618bb828f60 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:21:06.124021 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f921ef66-e316-47ba-a98b-e618bb828f60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:21:06.125935 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f921ef66-e316-47ba-a98b-e618bb828f60 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:21:06.132426 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:21:06.205762 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1330/2612] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:21:06 2022] POST /identity/v3/users => generated 286 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:21:06.209755 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b2d10-33d8-4bfc-8153-f03f439c1684 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:21:06.220692 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418b2d10-33d8-4bfc-8153-f03f439c1684 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:21:06.225467 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed4dbed-e828-49d4-b4ec-f1c1a76cbc3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:21:06.226129 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4dbed-e828-49d4-b4ec-f1c1a76cbc3e admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:21:06.226415 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4dbed-e828-49d4-b4ec-f1c1a76cbc3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:21:06.226700 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed4dbed-e828-49d4-b4ec-f1c1a76cbc3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:21:06.227438 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed4dbed-e828-49d4-b4ec-f1c1a76cbc3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:21:06.229076 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed4dbed-e828-49d4-b4ec-f1c1a76cbc3e admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:21:06.239679 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1283/2613] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:21:06.243711 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f921ef66-e316-47ba-a98b-e618bb828f60 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:21:06.253555 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f921ef66-e316-47ba-a98b-e618bb828f60 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:21:06.258113 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f487e470-6339-4455-bba7-cb258c00301d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:21:06.259326 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f487e470-6339-4455-bba7-cb258c00301d admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:21:06.259612 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f487e470-6339-4455-bba7-cb258c00301d admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:21:06.259932 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f487e470-6339-4455-bba7-cb258c00301d admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:21:06.260656 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f487e470-6339-4455-bba7-cb258c00301d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:21:06.262244 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f487e470-6339-4455-bba7-cb258c00301d admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:21:06.273108 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1331/2614] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:21:06.277329 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed4dbed-e828-49d4-b4ec-f1c1a76cbc3e admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:21:06.287356 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed4dbed-e828-49d4-b4ec-f1c1a76cbc3e admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:21:06.291895 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45569553-7d54-49d6-b1e0-8e04197ab1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:21:06.292762 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45569553-7d54-49d6-b1e0-8e04197ab1c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:21:06.293061 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45569553-7d54-49d6-b1e0-8e04197ab1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:21:06.293318 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45569553-7d54-49d6-b1e0-8e04197ab1c1 admin admin] PATH_INFO: `/v3/projects/1d7fa4c1086442019950404d5276f18c/users/a6be38a15e4e493ba7086ad7a2e29415/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:21:06.307285 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45569553-7d54-49d6-b1e0-8e04197ab1c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d7fa4c1086442019950404d5276f18c, user_id=a6be38a15e4e493ba7086ad7a2e29415, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:21:06.309915 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45569553-7d54-49d6-b1e0-8e04197ab1c1 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:21:06.409263 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1284/2615] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:21:06 2022] PUT /identity/v3/projects/1d7fa4c1086442019950404d5276f18c/users/a6be38a15e4e493ba7086ad7a2e29415/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:21:06.420140 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408fff7-9811-4860-b405-87c3d385c05a None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:21:06.420408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408fff7-9811-4860-b405-87c3d385c05a None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:21:06.420697 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4408fff7-9811-4860-b405-87c3d385c05a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:21:06.447942 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-4408fff7-9811-4860-b405-87c3d385c05a None None] MFA Rules not processed for user `a6be38a15e4e493ba7086ad7a2e29415`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:21:06.450806 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-4408fff7-9811-4860-b405-87c3d385c05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:21:06.512515 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1332/2616] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:21:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:21:06.516864 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45569553-7d54-49d6-b1e0-8e04197ab1c1 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:21:06.579641 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45569553-7d54-49d6-b1e0-8e04197ab1c1 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:21:06.584188 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a3137c-a497-4e58-8368-08c22dbd5bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:21:06.584843 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a3137c-a497-4e58-8368-08c22dbd5bb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:21:06.585127 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a3137c-a497-4e58-8368-08c22dbd5bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:21:06.585400 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a3137c-a497-4e58-8368-08c22dbd5bb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:21:06.586090 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a3137c-a497-4e58-8368-08c22dbd5bb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:21:06.588045 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a3137c-a497-4e58-8368-08c22dbd5bb2 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:21:06.715951 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1285/2617] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:21:06 2022] POST /identity/v3/projects => generated 373 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:21:06.719917 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4408fff7-9811-4860-b405-87c3d385c05a None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:21:06.733813 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4408fff7-9811-4860-b405-87c3d385c05a None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:21:06.794152 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204522df-55cb-41a1-a94c-d73635f9a77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:21:06.794866 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204522df-55cb-41a1-a94c-d73635f9a77f admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:21:06.795146 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204522df-55cb-41a1-a94c-d73635f9a77f admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:21:06.795423 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204522df-55cb-41a1-a94c-d73635f9a77f admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:21:06.796156 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204522df-55cb-41a1-a94c-d73635f9a77f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:21:06.798269 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204522df-55cb-41a1-a94c-d73635f9a77f admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:21:06.940159 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1333/2618] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:21:06 2022] POST /identity/v3/users => generated 293 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:21:06.944092 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a3137c-a497-4e58-8368-08c22dbd5bb2 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:21:06.954544 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a3137c-a497-4e58-8368-08c22dbd5bb2 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:21:06.959096 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e609b91-7575-4989-b9e0-a8e6a2159f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:21:06.959791 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e609b91-7575-4989-b9e0-a8e6a2159f6d admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:21:06.960081 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e609b91-7575-4989-b9e0-a8e6a2159f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:21:06.960363 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e609b91-7575-4989-b9e0-a8e6a2159f6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:21:06.961091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e609b91-7575-4989-b9e0-a8e6a2159f6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:21:06.962811 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e609b91-7575-4989-b9e0-a8e6a2159f6d admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:21:06.973412 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1286/2619] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:21:06.978094 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204522df-55cb-41a1-a94c-d73635f9a77f admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:21:06.988112 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204522df-55cb-41a1-a94c-d73635f9a77f admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:21:06.993500 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f45db2-b76b-43b9-966c-60f3b1829e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:21:06.994378 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f45db2-b76b-43b9-966c-60f3b1829e1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:21:06.994666 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f45db2-b76b-43b9-966c-60f3b1829e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:21:06.994938 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f45db2-b76b-43b9-966c-60f3b1829e1c admin admin] PATH_INFO: `/v3/projects/f76ba82c003146419310d2fcadc0cd6e/users/2ab0eae69a074048a1374456fcfecbe0/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:21:07.009395 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f45db2-b76b-43b9-966c-60f3b1829e1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f76ba82c003146419310d2fcadc0cd6e, user_id=2ab0eae69a074048a1374456fcfecbe0, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:21:07.012082 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f45db2-b76b-43b9-966c-60f3b1829e1c admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:21:07.134155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1334/2620] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:21:06 2022] PUT /identity/v3/projects/f76ba82c003146419310d2fcadc0cd6e/users/2ab0eae69a074048a1374456fcfecbe0/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:21:07.138141 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e609b91-7575-4989-b9e0-a8e6a2159f6d admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:21:07.148345 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e609b91-7575-4989-b9e0-a8e6a2159f6d admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:21:07.204618 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5c7ac-a35b-4429-b649-c319ad8132a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:21:07.205211 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5c7ac-a35b-4429-b649-c319ad8132a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:21:07.205494 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5c7ac-a35b-4429-b649-c319ad8132a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:21:07.205770 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5c7ac-a35b-4429-b649-c319ad8132a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:21:07.206483 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f5c7ac-a35b-4429-b649-c319ad8132a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:21:07.208153 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f5c7ac-a35b-4429-b649-c319ad8132a0 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:21:07.217726 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1287/2621] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:21:07 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:21:07.221878 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f45db2-b76b-43b9-966c-60f3b1829e1c admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:21:07.232580 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f45db2-b76b-43b9-966c-60f3b1829e1c admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:21:07.237179 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98560d45-7c7c-445e-87fa-408cf8f583e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:21:07.238053 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98560d45-7c7c-445e-87fa-408cf8f583e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:21:07.238335 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98560d45-7c7c-445e-87fa-408cf8f583e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:21:07.238612 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98560d45-7c7c-445e-87fa-408cf8f583e6 admin admin] PATH_INFO: `/v3/projects/f76ba82c003146419310d2fcadc0cd6e/users/2ab0eae69a074048a1374456fcfecbe0/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:21:07.240602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98560d45-7c7c-445e-87fa-408cf8f583e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f76ba82c003146419310d2fcadc0cd6e, user_id=2ab0eae69a074048a1374456fcfecbe0, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:21:07.243944 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98560d45-7c7c-445e-87fa-408cf8f583e6 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:21:07.253974 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1335/2622] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:21:07 2022] PUT /identity/v3/projects/f76ba82c003146419310d2fcadc0cd6e/users/2ab0eae69a074048a1374456fcfecbe0/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:21:07.263668 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77479118-d75f-45bc-8c4e-a957008abbbc None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:21:07.264037 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77479118-d75f-45bc-8c4e-a957008abbbc None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:21:07.264325 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77479118-d75f-45bc-8c4e-a957008abbbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:21:07.290426 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-77479118-d75f-45bc-8c4e-a957008abbbc None None] MFA Rules not processed for user `2ab0eae69a074048a1374456fcfecbe0`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:21:07.291615 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-77479118-d75f-45bc-8c4e-a957008abbbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:21:07.312691 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:21:07.361312 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1288/2623] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:21:07 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:21:07.372840 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98560d45-7c7c-445e-87fa-408cf8f583e6 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:21:07.374501 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-98560d45-7c7c-445e-87fa-408cf8f583e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:21:07.396691 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:21:07.432964 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98560d45-7c7c-445e-87fa-408cf8f583e6 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:21:07.436824 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab07724c-6c51-4a40-8d54-af12c9dc9884 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:21:07.437481 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab07724c-6c51-4a40-8d54-af12c9dc9884 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:21:07.437768 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab07724c-6c51-4a40-8d54-af12c9dc9884 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:21:07.438048 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab07724c-6c51-4a40-8d54-af12c9dc9884 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:21:07.447644 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab07724c-6c51-4a40-8d54-af12c9dc9884 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:21:07.449131 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab07724c-6c51-4a40-8d54-af12c9dc9884 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:21:07.458703 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1336/2624] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:21:07 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:21:07.640052 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77479118-d75f-45bc-8c4e-a957008abbbc None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:21:07.649988 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77479118-d75f-45bc-8c4e-a957008abbbc None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:21:07.654642 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ca884-5428-4a01-a034-09be77a69f27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:21:07.655279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ca884-5428-4a01-a034-09be77a69f27 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:21:07.655561 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ca884-5428-4a01-a034-09be77a69f27 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:21:07.655873 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ca884-5428-4a01-a034-09be77a69f27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:21:07.715658 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508ca884-5428-4a01-a034-09be77a69f27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:21:07.717137 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508ca884-5428-4a01-a034-09be77a69f27 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:21:07.726328 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1289/2625] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:21:07 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:21:17.546089 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=17 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:21:17.722081 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=48 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:22:03.014508 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab07724c-6c51-4a40-8d54-af12c9dc9884 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:22:03.015952 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-ab07724c-6c51-4a40-8d54-af12c9dc9884 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:22:03.088186 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab07724c-6c51-4a40-8d54-af12c9dc9884 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:22:03.093151 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb26f2eb-c3b3-450a-8fe9-800c3ff43328 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76ca877bee9041d28ad41e45ef3dd18c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:22:03.093795 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26f2eb-c3b3-450a-8fe9-800c3ff43328 service placement] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:22:03.094085 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26f2eb-c3b3-450a-8fe9-800c3ff43328 service placement] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:22:03.094873 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb26f2eb-c3b3-450a-8fe9-800c3ff43328 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:22:03.096643 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-bb26f2eb-c3b3-450a-8fe9-800c3ff43328 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:22:03.108316 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb26f2eb-c3b3-450a-8fe9-800c3ff43328 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:22:03.110555 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb26f2eb-c3b3-450a-8fe9-800c3ff43328 service placement] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:22:03.120956 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1337/2626] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:22:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:22:05.081261 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b5d5e5-3b2b-4ef1-94e2-16fc0983ee10 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:22:05.083219 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b5d5e5-3b2b-4ef1-94e2-16fc0983ee10 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:22:05.083219 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b5d5e5-3b2b-4ef1-94e2-16fc0983ee10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:22:05.100293 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-d3b5d5e5-3b2b-4ef1-94e2-16fc0983ee10 None None] MFA Rules not processed for user `ae705e3500914acda9d7a123b4c36298`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:22:05.101532 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d3b5d5e5-3b2b-4ef1-94e2-16fc0983ee10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:22:05.154243 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1290/2627] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:22:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:22:13.116518 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=32 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:22:15.147706 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=25 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:24:20.967528 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb26f2eb-c3b3-450a-8fe9-800c3ff43328 service placement] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:24:20.969065 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-bb26f2eb-c3b3-450a-8fe9-800c3ff43328 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:24:21.025118 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb26f2eb-c3b3-450a-8fe9-800c3ff43328 service placement] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:24:21.028582 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638563f6-782e-475a-944d-387c20d1dcf9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:24:21.029223 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638563f6-782e-475a-944d-387c20d1dcf9 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:24:21.029506 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638563f6-782e-475a-944d-387c20d1dcf9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:24:21.029781 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638563f6-782e-475a-944d-387c20d1dcf9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:24:21.031491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-638563f6-782e-475a-944d-387c20d1dcf9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:24:21.041633 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638563f6-782e-475a-944d-387c20d1dcf9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:24:21.043045 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638563f6-782e-475a-944d-387c20d1dcf9 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:24:21.050779 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1338/2628] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:24:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:24:29.747112 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b5d5e5-3b2b-4ef1-94e2-16fc0983ee10 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:24:29.748540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-d3b5d5e5-3b2b-4ef1-94e2-16fc0983ee10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:24:29.760254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b5d5e5-3b2b-4ef1-94e2-16fc0983ee10 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:24:29.763718 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa57b2-9fd3-4336-b008-10ec774a82d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:24:29.764378 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa57b2-9fd3-4336-b008-10ec774a82d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:24:29.764660 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa57b2-9fd3-4336-b008-10ec774a82d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:24:29.764932 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defa57b2-9fd3-4336-b008-10ec774a82d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:24:29.766594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-defa57b2-9fd3-4336-b008-10ec774a82d6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:24:29.825351 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defa57b2-9fd3-4336-b008-10ec774a82d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:24:29.826795 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defa57b2-9fd3-4336-b008-10ec774a82d6 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:24:29.836452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1291/2629] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:24:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:24:31.047305 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=26 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:24:39.831759 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=28 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:24:50.813373 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638563f6-782e-475a-944d-387c20d1dcf9 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:24:50.814795 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-638563f6-782e-475a-944d-387c20d1dcf9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:24:50.887773 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638563f6-782e-475a-944d-387c20d1dcf9 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:24:50.893184 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2cf162-8c68-4d19-91f1-31b8608615eb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:24:50.893823 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2cf162-8c68-4d19-91f1-31b8608615eb service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:24:50.894107 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2cf162-8c68-4d19-91f1-31b8608615eb service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:24:50.894380 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2cf162-8c68-4d19-91f1-31b8608615eb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:24:50.896142 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-ab2cf162-8c68-4d19-91f1-31b8608615eb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:24:50.911795 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2cf162-8c68-4d19-91f1-31b8608615eb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:24:50.913377 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2cf162-8c68-4d19-91f1-31b8608615eb service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:24:50.922204 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1339/2630] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:24:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:25:00.918168 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=34 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:26:07.411564 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa57b2-9fd3-4336-b008-10ec774a82d6 service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:26:07.430189 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defa57b2-9fd3-4336-b008-10ec774a82d6 service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:26:07.433854 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a919e9d-fee6-4949-8aeb-04b384e0479c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:26:07.434492 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a919e9d-fee6-4949-8aeb-04b384e0479c service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:26:07.434776 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a919e9d-fee6-4949-8aeb-04b384e0479c service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:26:07.435054 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a919e9d-fee6-4949-8aeb-04b384e0479c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:26:07.456926 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a919e9d-fee6-4949-8aeb-04b384e0479c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:26:07.458342 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a919e9d-fee6-4949-8aeb-04b384e0479c service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:26:07.467043 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1292/2631] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:26:07 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:26:17.463056 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=17 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:27:13.182524 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2cf162-8c68-4d19-91f1-31b8608615eb service nova] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:27:13.201190 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2cf162-8c68-4d19-91f1-31b8608615eb service nova] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:27:13.204765 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f2c879-3b72-4e69-bc52-d7cd7de55629 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:27:13.205487 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f2c879-3b72-4e69-bc52-d7cd7de55629 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:27:13.205775 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f2c879-3b72-4e69-bc52-d7cd7de55629 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:27:13.206054 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f2c879-3b72-4e69-bc52-d7cd7de55629 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:27:13.207938 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b4f2c879-3b72-4e69-bc52-d7cd7de55629 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:27:13.265594 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f2c879-3b72-4e69-bc52-d7cd7de55629 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:27:13.267024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f2c879-3b72-4e69-bc52-d7cd7de55629 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:27:13.287718 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1340/2632] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:27:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:27:13.550079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a919e9d-fee6-4949-8aeb-04b384e0479c service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:27:13.559539 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a919e9d-fee6-4949-8aeb-04b384e0479c service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:27:13.563396 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca0ed2-d85c-464d-9a2c-77db827932c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:27:13.564066 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca0ed2-d85c-464d-9a2c-77db827932c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:27:13.564353 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca0ed2-d85c-464d-9a2c-77db827932c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:27:13.564632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca0ed2-d85c-464d-9a2c-77db827932c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:27:13.573632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca0ed2-d85c-464d-9a2c-77db827932c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:27:13.577692 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca0ed2-d85c-464d-9a2c-77db827932c0 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:27:13.584616 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1293/2633] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:27:13 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:27:23.285125 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=34 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:27:23.580351 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=6 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:27:30.175038 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f2c879-3b72-4e69-bc52-d7cd7de55629 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:27:30.176444 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b4f2c879-3b72-4e69-bc52-d7cd7de55629 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:27:30.212287 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f2c879-3b72-4e69-bc52-d7cd7de55629 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:27:30.217227 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6ed68a-a88e-48a7-91c9-18f1b2abdc9d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76ca877bee9041d28ad41e45ef3dd18c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:27:30.217864 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ed68a-a88e-48a7-91c9-18f1b2abdc9d service placement] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:27:30.218145 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ed68a-a88e-48a7-91c9-18f1b2abdc9d service placement] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:27:30.218419 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ed68a-a88e-48a7-91c9-18f1b2abdc9d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:27:30.227484 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6ed68a-a88e-48a7-91c9-18f1b2abdc9d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:27:30.229089 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6ed68a-a88e-48a7-91c9-18f1b2abdc9d service placement] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:27:30.238862 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1341/2634] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:27:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:27:40.234618 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=15 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:29:24.936193 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca0ed2-d85c-464d-9a2c-77db827932c0 service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:29:24.954761 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca0ed2-d85c-464d-9a2c-77db827932c0 service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:29:24.958287 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb5a7ef-d18f-489e-8677-8da7725dd11c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:29:24.958916 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5a7ef-d18f-489e-8677-8da7725dd11c service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:29:24.959223 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5a7ef-d18f-489e-8677-8da7725dd11c service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:29:24.959499 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5a7ef-d18f-489e-8677-8da7725dd11c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:29:24.966614 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb5a7ef-d18f-489e-8677-8da7725dd11c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:29:24.968058 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb5a7ef-d18f-489e-8677-8da7725dd11c service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:29:24.977087 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1294/2635] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:29:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:29:29.415134 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6ed68a-a88e-48a7-91c9-18f1b2abdc9d service placement] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:29:29.434533 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6ed68a-a88e-48a7-91c9-18f1b2abdc9d service placement] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:29:29.438964 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf815e8-af46-4124-ae28-97c0f4d066d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:29:29.439608 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf815e8-af46-4124-ae28-97c0f4d066d7 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:29:29.439933 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf815e8-af46-4124-ae28-97c0f4d066d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:29:29.440211 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf815e8-af46-4124-ae28-97c0f4d066d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:29:29.449970 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf815e8-af46-4124-ae28-97c0f4d066d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:29:29.451448 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf815e8-af46-4124-ae28-97c0f4d066d7 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:29:29.461585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1342/2636] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:29:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:29:34.972913 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=10 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:29:39.457011 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=10 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:29:51.875842 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb5a7ef-d18f-489e-8677-8da7725dd11c service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:29:51.885654 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb5a7ef-d18f-489e-8677-8da7725dd11c service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:29:51.889206 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0e06a-b9ce-48dc-b061-e5f8cdb7e778 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:29:51.889835 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0e06a-b9ce-48dc-b061-e5f8cdb7e778 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:29:51.890120 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0e06a-b9ce-48dc-b061-e5f8cdb7e778 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:29:51.890397 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0e06a-b9ce-48dc-b061-e5f8cdb7e778 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:29:51.899277 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b0e06a-b9ce-48dc-b061-e5f8cdb7e778 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:29:51.900721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b0e06a-b9ce-48dc-b061-e5f8cdb7e778 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:29:51.927025 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1295/2637] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:29:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:30:01.924465 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=13 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:31:07.614472 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf815e8-af46-4124-ae28-97c0f4d066d7 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:31:07.616709 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-8bf815e8-af46-4124-ae28-97c0f4d066d7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:31:07.684271 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf815e8-af46-4124-ae28-97c0f4d066d7 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:31:07.687858 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9bb209-f778-42be-a0b8-fea62e60b22d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:31:07.688495 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9bb209-f778-42be-a0b8-fea62e60b22d service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:31:07.688773 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9bb209-f778-42be-a0b8-fea62e60b22d service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:31:07.689045 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9bb209-f778-42be-a0b8-fea62e60b22d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:31:07.690873 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-cf9bb209-f778-42be-a0b8-fea62e60b22d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:31:07.767679 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9bb209-f778-42be-a0b8-fea62e60b22d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:31:07.769145 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9bb209-f778-42be-a0b8-fea62e60b22d service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:31:07.787992 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1343/2638] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:31:07 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:31:17.784526 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=66 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:32:32.169003 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0e06a-b9ce-48dc-b061-e5f8cdb7e778 service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:32:32.170598 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-78b0e06a-b9ce-48dc-b061-e5f8cdb7e778 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:32:32.257614 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0e06a-b9ce-48dc-b061-e5f8cdb7e778 service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:32:32.263580 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02748139-748b-49b3-b1be-db91188d63b0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76ca877bee9041d28ad41e45ef3dd18c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:32:32.277145 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02748139-748b-49b3-b1be-db91188d63b0 service placement] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:32:32.277145 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02748139-748b-49b3-b1be-db91188d63b0 service placement] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:32:32.277145 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02748139-748b-49b3-b1be-db91188d63b0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:32:32.277145 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-02748139-748b-49b3-b1be-db91188d63b0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:32:32.281840 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02748139-748b-49b3-b1be-db91188d63b0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:32:32.283443 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02748139-748b-49b3-b1be-db91188d63b0 service placement] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:32:32.294144 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1296/2639] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:32:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:32:42.289581 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=32 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:33:14.268333 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9bb209-f778-42be-a0b8-fea62e60b22d service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:33:14.269858 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-cf9bb209-f778-42be-a0b8-fea62e60b22d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:33:14.296044 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9bb209-f778-42be-a0b8-fea62e60b22d service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:33:14.299648 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f397f-51c6-4208-bf72-144fd953bf5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:33:14.305141 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f397f-51c6-4208-bf72-144fd953bf5c service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:33:14.305141 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f397f-51c6-4208-bf72-144fd953bf5c service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:33:14.305141 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880f397f-51c6-4208-bf72-144fd953bf5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:33:14.311952 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f397f-51c6-4208-bf72-144fd953bf5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:33:14.313361 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880f397f-51c6-4208-bf72-144fd953bf5c service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:33:14.324137 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1344/2640] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:33:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:33:15.503634 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02748139-748b-49b3-b1be-db91188d63b0 service placement] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:33:15.513042 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02748139-748b-49b3-b1be-db91188d63b0 service placement] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:33:15.517330 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461b7283-8764-4a86-bb39-fa6ae2c90995 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:33:15.517967 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b7283-8764-4a86-bb39-fa6ae2c90995 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:33:15.518258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b7283-8764-4a86-bb39-fa6ae2c90995 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:33:15.518539 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b7283-8764-4a86-bb39-fa6ae2c90995 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:33:15.520335 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-461b7283-8764-4a86-bb39-fa6ae2c90995 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:33:15.533373 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461b7283-8764-4a86-bb39-fa6ae2c90995 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:33:15.534803 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461b7283-8764-4a86-bb39-fa6ae2c90995 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:33:15.545925 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1297/2641] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:33:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:33:24.319320 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=11 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:33:25.541066 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=6 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:34:29.132794 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f397f-51c6-4208-bf72-144fd953bf5c service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:34:29.185763 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880f397f-51c6-4208-bf72-144fd953bf5c service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:34:29.190114 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e227634-760b-4bfc-a7f5-cf5a53611e86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:34:29.190757 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e227634-760b-4bfc-a7f5-cf5a53611e86 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:34:29.191038 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e227634-760b-4bfc-a7f5-cf5a53611e86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:34:29.191311 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e227634-760b-4bfc-a7f5-cf5a53611e86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:34:29.199964 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e227634-760b-4bfc-a7f5-cf5a53611e86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:34:29.201399 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e227634-760b-4bfc-a7f5-cf5a53611e86 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:34:29.234789 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1345/2642] 10.222.0.40 () {60 vars in 1396 bytes} [Wed Jul 6 17:34:29 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3250 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:34:30.941144 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461b7283-8764-4a86-bb39-fa6ae2c90995 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:34:30.942720 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-461b7283-8764-4a86-bb39-fa6ae2c90995 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:34:30.995768 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461b7283-8764-4a86-bb39-fa6ae2c90995 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:34:30.999771 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef00295-e24f-4326-96f8-f7a7c7891aa4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:34:31.000406 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef00295-e24f-4326-96f8-f7a7c7891aa4 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:34:31.000688 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef00295-e24f-4326-96f8-f7a7c7891aa4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:34:31.000962 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef00295-e24f-4326-96f8-f7a7c7891aa4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:34:31.002822 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-aef00295-e24f-4326-96f8-f7a7c7891aa4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:34:31.012544 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef00295-e24f-4326-96f8-f7a7c7891aa4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:34:31.013987 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef00295-e24f-4326-96f8-f7a7c7891aa4 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:34:31.022846 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1298/2643] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:34:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:34:39.231323 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=33 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:34:41.019201 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=26 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:34:51.540579 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e227634-760b-4bfc-a7f5-cf5a53611e86 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:34:51.542203 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-2e227634-760b-4bfc-a7f5-cf5a53611e86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:34:51.554542 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e227634-760b-4bfc-a7f5-cf5a53611e86 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:34:51.558152 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3d772-63dc-4d51-91b8-7fb33ff4179f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:34:51.558818 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3d772-63dc-4d51-91b8-7fb33ff4179f service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:34:51.559105 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3d772-63dc-4d51-91b8-7fb33ff4179f service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:34:51.559386 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a3d772-63dc-4d51-91b8-7fb33ff4179f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:34:51.561230 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c0a3d772-63dc-4d51-91b8-7fb33ff4179f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:34:51.630401 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a3d772-63dc-4d51-91b8-7fb33ff4179f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:34:51.631843 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a3d772-63dc-4d51-91b8-7fb33ff4179f service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:34:51.640621 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1346/2644] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:34:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:35:01.636601 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=33 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:35:04.027218 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef00295-e24f-4326-96f8-f7a7c7891aa4 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:35:04.027796 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-aef00295-e24f-4326-96f8-f7a7c7891aa4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:35:04.135183 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef00295-e24f-4326-96f8-f7a7c7891aa4 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:35:04.142527 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7351548-a2d1-4fd8-b33d-a2937125c979 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:35:04.143211 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7351548-a2d1-4fd8-b33d-a2937125c979 service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:35:04.143496 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7351548-a2d1-4fd8-b33d-a2937125c979 service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:35:04.143798 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7351548-a2d1-4fd8-b33d-a2937125c979 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:35:04.146612 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c7351548-a2d1-4fd8-b33d-a2937125c979 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:35:04.162417 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7351548-a2d1-4fd8-b33d-a2937125c979 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:35:04.167464 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7351548-a2d1-4fd8-b33d-a2937125c979 service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:35:04.185903 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1299/2645] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:35:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:35:14.176585 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=34 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:37:02.941032 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3d772-63dc-4d51-91b8-7fb33ff4179f service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:37:02.949987 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a3d772-63dc-4d51-91b8-7fb33ff4179f service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:37:02.953787 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd2877a-7a6b-4001-9158-844da7b1cfa5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:37:02.954431 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2877a-7a6b-4001-9158-844da7b1cfa5 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:37:02.954717 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2877a-7a6b-4001-9158-844da7b1cfa5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:37:02.954995 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2877a-7a6b-4001-9158-844da7b1cfa5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:37:02.976123 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd2877a-7a6b-4001-9158-844da7b1cfa5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:37:02.977550 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd2877a-7a6b-4001-9158-844da7b1cfa5 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:37:02.986819 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1347/2646] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:37:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:37:12.982315 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=12 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:37:35.131686 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7351548-a2d1-4fd8-b33d-a2937125c979 service nova] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:37:35.133582 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c7351548-a2d1-4fd8-b33d-a2937125c979 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:37:35.217325 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7351548-a2d1-4fd8-b33d-a2937125c979 service nova] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:37:35.222975 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1225f2e2-fcc4-4f5b-9b9c-678b1182eb06 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76ca877bee9041d28ad41e45ef3dd18c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:37:35.223624 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225f2e2-fcc4-4f5b-9b9c-678b1182eb06 service placement] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:37:35.223948 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225f2e2-fcc4-4f5b-9b9c-678b1182eb06 service placement] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:37:35.224235 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1225f2e2-fcc4-4f5b-9b9c-678b1182eb06 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:37:35.232828 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1225f2e2-fcc4-4f5b-9b9c-678b1182eb06 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:37:35.234968 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1225f2e2-fcc4-4f5b-9b9c-678b1182eb06 service placement] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:37:35.244916 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1300/2647] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:37:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 17:37:45.240614 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=37 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:38:14.500836 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd2877a-7a6b-4001-9158-844da7b1cfa5 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:38:14.519909 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd2877a-7a6b-4001-9158-844da7b1cfa5 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:38:14.523452 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd00e7-afab-4a91-b3d8-425ea0358474 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:38:14.524117 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd00e7-afab-4a91-b3d8-425ea0358474 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:38:14.524402 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd00e7-afab-4a91-b3d8-425ea0358474 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:38:14.524676 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cd00e7-afab-4a91-b3d8-425ea0358474 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:38:14.526498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-56cd00e7-afab-4a91-b3d8-425ea0358474 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:38:14.561478 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cd00e7-afab-4a91-b3d8-425ea0358474 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:38:14.562907 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cd00e7-afab-4a91-b3d8-425ea0358474 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:38:14.583731 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1348/2648] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:38:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:38:16.011957 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1225f2e2-fcc4-4f5b-9b9c-678b1182eb06 service placement] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:38:16.021056 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1225f2e2-fcc4-4f5b-9b9c-678b1182eb06 service placement] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:38:16.024640 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2441a795-dd14-4b2d-8fb4-172817d5c5da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:38:16.025270 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2441a795-dd14-4b2d-8fb4-172817d5c5da service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:38:16.025556 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2441a795-dd14-4b2d-8fb4-172817d5c5da service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:38:16.025834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2441a795-dd14-4b2d-8fb4-172817d5c5da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:38:16.035846 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2441a795-dd14-4b2d-8fb4-172817d5c5da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:38:16.037529 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2441a795-dd14-4b2d-8fb4-172817d5c5da service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:38:16.048313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1301/2649] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:38:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:38:24.580492 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=23 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:38:26.043564 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=6 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:39:51.596746 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd00e7-afab-4a91-b3d8-425ea0358474 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:39:51.607841 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cd00e7-afab-4a91-b3d8-425ea0358474 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:39:51.613823 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca0e69-9e48-4b10-9bf8-f1a634bf2c67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:39:51.614473 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca0e69-9e48-4b10-9bf8-f1a634bf2c67 service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:39:51.614741 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca0e69-9e48-4b10-9bf8-f1a634bf2c67 service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:39:51.615014 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ca0e69-9e48-4b10-9bf8-f1a634bf2c67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:39:51.622769 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ca0e69-9e48-4b10-9bf8-f1a634bf2c67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:39:51.624384 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ca0e69-9e48-4b10-9bf8-f1a634bf2c67 service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:39:51.633188 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1349/2650] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:39:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:40:01.629175 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=6 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:41:03.346394 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2441a795-dd14-4b2d-8fb4-172817d5c5da service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:41:03.368559 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2441a795-dd14-4b2d-8fb4-172817d5c5da service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:41:03.368932 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b96d7a-90ed-4e08-b072-a3de233f3fd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:41:03.370104 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b96d7a-90ed-4e08-b072-a3de233f3fd8 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:41:03.370390 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b96d7a-90ed-4e08-b072-a3de233f3fd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:41:03.370671 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b96d7a-90ed-4e08-b072-a3de233f3fd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:41:03.372592 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-38b96d7a-90ed-4e08-b072-a3de233f3fd8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:41:03.458862 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b96d7a-90ed-4e08-b072-a3de233f3fd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:41:03.460609 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b96d7a-90ed-4e08-b072-a3de233f3fd8 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:41:03.469993 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1302/2651] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:41:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:41:04.578726 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca0e69-9e48-4b10-9bf8-f1a634bf2c67 service nova] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:41:04.588855 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ca0e69-9e48-4b10-9bf8-f1a634bf2c67 service nova] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:41:04.592636 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb0a02-96de-4732-9942-49e795fa6891 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:41:04.593271 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb0a02-96de-4732-9942-49e795fa6891 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:41:04.593554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb0a02-96de-4732-9942-49e795fa6891 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:41:04.593831 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb0a02-96de-4732-9942-49e795fa6891 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:41:04.595673 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-cedb0a02-96de-4732-9942-49e795fa6891 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:41:04.682629 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedb0a02-96de-4732-9942-49e795fa6891 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:41:04.684254 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedb0a02-96de-4732-9942-49e795fa6891 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:41:04.715890 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1350/2652] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:41:04 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:41:06.717821 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7878617-dd15-4481-9332-c4c01b7fea02 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:41:06.718334 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7878617-dd15-4481-9332-c4c01b7fea02 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:41:06.718454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7878617-dd15-4481-9332-c4c01b7fea02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:41:06.753710 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-f7878617-dd15-4481-9332-c4c01b7fea02 None None] MFA Rules not processed for user `ae705e3500914acda9d7a123b4c36298`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:41:06.754915 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-f7878617-dd15-4481-9332-c4c01b7fea02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:41:06.771099 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1303/2653] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:41:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:41:07.063191 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb0a02-96de-4732-9942-49e795fa6891 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:41:07.082323 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb0a02-96de-4732-9942-49e795fa6891 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:41:07.085929 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c42e7-f3b7-4cc8-a603-ea6a4372796a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:41:07.086568 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c42e7-f3b7-4cc8-a603-ea6a4372796a service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:41:07.086853 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c42e7-f3b7-4cc8-a603-ea6a4372796a service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:41:07.087131 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c42e7-f3b7-4cc8-a603-ea6a4372796a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:41:07.088939 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-152c42e7-f3b7-4cc8-a603-ea6a4372796a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:41:07.098183 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152c42e7-f3b7-4cc8-a603-ea6a4372796a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:41:07.099591 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152c42e7-f3b7-4cc8-a603-ea6a4372796a service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:41:07.107428 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1351/2654] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:41:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:41:07.549210 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7878617-dd15-4481-9332-c4c01b7fea02 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:41:07.562333 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7878617-dd15-4481-9332-c4c01b7fea02 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:41:07.567519 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9f9cac-0f13-4784-a766-746b53f9c798 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76ca877bee9041d28ad41e45ef3dd18c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:41:07.568243 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9f9cac-0f13-4784-a766-746b53f9c798 service placement] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:41:07.568533 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9f9cac-0f13-4784-a766-746b53f9c798 service placement] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:41:07.568810 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9f9cac-0f13-4784-a766-746b53f9c798 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:41:07.570582 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-8c9f9cac-0f13-4784-a766-746b53f9c798 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:41:07.582982 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9f9cac-0f13-4784-a766-746b53f9c798 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:41:07.584423 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9f9cac-0f13-4784-a766-746b53f9c798 service placement] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:41:07.594873 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1304/2655] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:41:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:41:10.257396 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c42e7-f3b7-4cc8-a603-ea6a4372796a service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:41:10.268037 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c42e7-f3b7-4cc8-a603-ea6a4372796a service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:41:10.271863 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f948c620-be04-43e9-87f0-e097d6cf1f4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:41:10.272518 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948c620-be04-43e9-87f0-e097d6cf1f4b service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:41:10.272820 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948c620-be04-43e9-87f0-e097d6cf1f4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:41:10.273100 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f948c620-be04-43e9-87f0-e097d6cf1f4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:41:10.280963 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f948c620-be04-43e9-87f0-e097d6cf1f4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:41:10.282375 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f948c620-be04-43e9-87f0-e097d6cf1f4b service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:41:10.290488 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1352/2656] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:41:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:41:11.123223 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9f9cac-0f13-4784-a766-746b53f9c798 service placement] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:41:11.128560 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-8c9f9cac-0f13-4784-a766-746b53f9c798 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:41:11.141217 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9f9cac-0f13-4784-a766-746b53f9c798 service placement] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:41:11.147994 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff9faa6-6b6d-4fdc-87aa-cb6e74b3a12d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:41:11.148653 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff9faa6-6b6d-4fdc-87aa-cb6e74b3a12d service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:41:11.148938 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff9faa6-6b6d-4fdc-87aa-cb6e74b3a12d service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:41:11.149212 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff9faa6-6b6d-4fdc-87aa-cb6e74b3a12d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:41:11.159720 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff9faa6-6b6d-4fdc-87aa-cb6e74b3a12d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:41:11.161501 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff9faa6-6b6d-4fdc-87aa-cb6e74b3a12d service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:41:11.173344 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1305/2657] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:41:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 17:41:16.016472 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f948c620-be04-43e9-87f0-e097d6cf1f4b service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:41:16.029243 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f948c620-be04-43e9-87f0-e097d6cf1f4b service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:41:16.034693 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f2da4-f0d6-46c3-9e90-fd5f9d970a61 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:41:16.035328 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f2da4-f0d6-46c3-9e90-fd5f9d970a61 service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:41:16.035613 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f2da4-f0d6-46c3-9e90-fd5f9d970a61 service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:41:16.035931 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f2da4-f0d6-46c3-9e90-fd5f9d970a61 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:41:16.037949 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-a62f2da4-f0d6-46c3-9e90-fd5f9d970a61 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:41:16.049754 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f2da4-f0d6-46c3-9e90-fd5f9d970a61 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:41:16.051306 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f2da4-f0d6-46c3-9e90-fd5f9d970a61 service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:41:16.062147 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1353/2658] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:41:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:41:19.109391 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff9faa6-6b6d-4fdc-87aa-cb6e74b3a12d service nova] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:41:19.121483 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff9faa6-6b6d-4fdc-87aa-cb6e74b3a12d service nova] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:41:19.127003 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cfc541-5311-4307-8486-068fad71ff2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:41:19.127738 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cfc541-5311-4307-8486-068fad71ff2d service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:41:19.128102 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cfc541-5311-4307-8486-068fad71ff2d service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:41:19.128450 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cfc541-5311-4307-8486-068fad71ff2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:41:19.130293 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-84cfc541-5311-4307-8486-068fad71ff2d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:41:19.143329 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cfc541-5311-4307-8486-068fad71ff2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:41:19.145035 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cfc541-5311-4307-8486-068fad71ff2d service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:41:19.155773 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1306/2659] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:41:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:41:26.057275 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=65 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:41:29.151184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=69 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:42:41.158570 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f2da4-f0d6-46c3-9e90-fd5f9d970a61 service nova] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:42:41.160774 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-a62f2da4-f0d6-46c3-9e90-fd5f9d970a61 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:42:41.227934 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f2da4-f0d6-46c3-9e90-fd5f9d970a61 service nova] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:42:41.233187 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f9633c-e6df-4acc-a28c-e12eccfa0b80 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76ca877bee9041d28ad41e45ef3dd18c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:42:41.233828 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f9633c-e6df-4acc-a28c-e12eccfa0b80 service placement] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:42:41.234109 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f9633c-e6df-4acc-a28c-e12eccfa0b80 service placement] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:42:41.234382 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f9633c-e6df-4acc-a28c-e12eccfa0b80 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:42:41.236268 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d1f9633c-e6df-4acc-a28c-e12eccfa0b80 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:42:41.247825 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f9633c-e6df-4acc-a28c-e12eccfa0b80 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:42:41.249456 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f9633c-e6df-4acc-a28c-e12eccfa0b80 service placement] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:42:41.259804 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1354/2660] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:42:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:42:51.255643 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=32 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:43:02.453298 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14390063-d0e8-42e6-97d4-fba0a8444e4b None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:43:02.453812 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14390063-d0e8-42e6-97d4-fba0a8444e4b None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:43:02.453918 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14390063-d0e8-42e6-97d4-fba0a8444e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:43:02.472046 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-14390063-d0e8-42e6-97d4-fba0a8444e4b None None] MFA Rules not processed for user `ae705e3500914acda9d7a123b4c36298`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:43:02.473251 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-14390063-d0e8-42e6-97d4-fba0a8444e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:43:02.480191 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1307/2661] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:43:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:43:04.819924 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f9633c-e6df-4acc-a28c-e12eccfa0b80 service placement] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:43:04.879117 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f9633c-e6df-4acc-a28c-e12eccfa0b80 service placement] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:43:04.882696 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc136e1f-737c-49c2-b2b4-a808290f7cac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:43:04.883342 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc136e1f-737c-49c2-b2b4-a808290f7cac service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:43:04.883629 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc136e1f-737c-49c2-b2b4-a808290f7cac service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:43:04.883997 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc136e1f-737c-49c2-b2b4-a808290f7cac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:43:04.885873 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-fc136e1f-737c-49c2-b2b4-a808290f7cac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:43:04.965027 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc136e1f-737c-49c2-b2b4-a808290f7cac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:43:04.966459 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc136e1f-737c-49c2-b2b4-a808290f7cac service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:43:04.984557 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1355/2662] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:43:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:43:05.336015 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14390063-d0e8-42e6-97d4-fba0a8444e4b None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:43:05.337767 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-14390063-d0e8-42e6-97d4-fba0a8444e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:43:05.458623 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14390063-d0e8-42e6-97d4-fba0a8444e4b None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:43:05.463237 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7d3485-4494-477c-9a07-c98ebf10e946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:43:05.465207 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d3485-4494-477c-9a07-c98ebf10e946 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:43:05.465207 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d3485-4494-477c-9a07-c98ebf10e946 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:43:05.465207 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7d3485-4494-477c-9a07-c98ebf10e946 admin admin] PATH_INFO: `/v3/users/a6be38a15e4e493ba7086ad7a2e29415` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:43:05.466313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7d3485-4494-477c-9a07-c98ebf10e946 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6be38a15e4e493ba7086ad7a2e29415)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:43:05.468285 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7d3485-4494-477c-9a07-c98ebf10e946 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:43:05.758243 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-9f7d3485-4494-477c-9a07-c98ebf10e946 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6be38a15e4e493ba7086ad7a2e29415'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:43:05.771887 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-9f7d3485-4494-477c-9a07-c98ebf10e946 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6be38a15e4e493ba7086ad7a2e29415'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:43:05.780540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-9f7d3485-4494-477c-9a07-c98ebf10e946 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6be38a15e4e493ba7086ad7a2e29415'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:43:05.781288 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-9f7d3485-4494-477c-9a07-c98ebf10e946 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6be38a15e4e493ba7086ad7a2e29415'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:43:05.838853 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-9f7d3485-4494-477c-9a07-c98ebf10e946 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6be38a15e4e493ba7086ad7a2e29415'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:43:05.846694 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1308/2663] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:43:05 2022] DELETE /identity/v3/users/a6be38a15e4e493ba7086ad7a2e29415 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:43:05.851470 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411108fd-2d12-42dc-bd25-daa21f9966b8 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:43:05.852075 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411108fd-2d12-42dc-bd25-daa21f9966b8 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:43:05.852075 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411108fd-2d12-42dc-bd25-daa21f9966b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:43:05.880260 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-411108fd-2d12-42dc-bd25-daa21f9966b8 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:43:05.881481 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-411108fd-2d12-42dc-bd25-daa21f9966b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:43:05.948650 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1356/2664] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:43:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:43:05.959665 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7d3485-4494-477c-9a07-c98ebf10e946 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:43:05.961051 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-9f7d3485-4494-477c-9a07-c98ebf10e946 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:43:06.009539 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:43:06.022526 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7d3485-4494-477c-9a07-c98ebf10e946 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:43:06.026237 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc30d7-1d23-4498-8a97-4a2843aa6b17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:43:06.031033 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc30d7-1d23-4498-8a97-4a2843aa6b17 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:43:06.031033 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc30d7-1d23-4498-8a97-4a2843aa6b17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:43:06.031033 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edc30d7-1d23-4498-8a97-4a2843aa6b17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:43:06.038224 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edc30d7-1d23-4498-8a97-4a2843aa6b17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:43:06.040326 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edc30d7-1d23-4498-8a97-4a2843aa6b17 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:43:06.054987 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1309/2665] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:43:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:43:06.581255 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411108fd-2d12-42dc-bd25-daa21f9966b8 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:43:06.583318 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-411108fd-2d12-42dc-bd25-daa21f9966b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:43:06.601901 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411108fd-2d12-42dc-bd25-daa21f9966b8 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:43:06.608258 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44e07d-414b-4391-8290-118aba98d815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:43:06.610847 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44e07d-414b-4391-8290-118aba98d815 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:43:06.611277 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44e07d-414b-4391-8290-118aba98d815 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:43:06.612080 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44e07d-414b-4391-8290-118aba98d815 admin admin] PATH_INFO: `/v3/projects/1d7fa4c1086442019950404d5276f18c` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:43:06.613524 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b44e07d-414b-4391-8290-118aba98d815 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d7fa4c1086442019950404d5276f18c)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:43:06.615463 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b44e07d-414b-4391-8290-118aba98d815 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:43:06.634089 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=100 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:43:06.792503 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8b44e07d-414b-4391-8290-118aba98d815 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d7fa4c1086442019950404d5276f18c', 'request_id': 'req-8b44e07d-414b-4391-8290-118aba98d815'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:43:06.801245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8b44e07d-414b-4391-8290-118aba98d815 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d7fa4c1086442019950404d5276f18c', 'request_id': 'req-8b44e07d-414b-4391-8290-118aba98d815'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:43:06.807624 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8b44e07d-414b-4391-8290-118aba98d815 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d7fa4c1086442019950404d5276f18c', 'request_id': 'req-8b44e07d-414b-4391-8290-118aba98d815'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:43:06.881854 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8b44e07d-414b-4391-8290-118aba98d815 admin admin] The token cache is being invalidate because project 1d7fa4c1086442019950404d5276f18c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:43:06.882518 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8b44e07d-414b-4391-8290-118aba98d815 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:43:06.882774 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-8b44e07d-414b-4391-8290-118aba98d815 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:43:06.884373 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1357/2666] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:43:06 2022] DELETE /identity/v3/projects/1d7fa4c1086442019950404d5276f18c => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:43:06.892359 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc30d7-1d23-4498-8a97-4a2843aa6b17 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:43:06.902169 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edc30d7-1d23-4498-8a97-4a2843aa6b17 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:43:06.959527 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37805b2-5d49-4415-8df1-a3cdd6d1bf2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:43:06.960463 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37805b2-5d49-4415-8df1-a3cdd6d1bf2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:43:06.960748 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37805b2-5d49-4415-8df1-a3cdd6d1bf2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:43:06.961025 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37805b2-5d49-4415-8df1-a3cdd6d1bf2a admin admin] PATH_INFO: `/v3/users/2ab0eae69a074048a1374456fcfecbe0` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:43:06.962219 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37805b2-5d49-4415-8df1-a3cdd6d1bf2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ab0eae69a074048a1374456fcfecbe0)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:43:06.964166 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37805b2-5d49-4415-8df1-a3cdd6d1bf2a admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:43:07.137079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-c37805b2-5d49-4415-8df1-a3cdd6d1bf2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ab0eae69a074048a1374456fcfecbe0'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:43:07.149167 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-c37805b2-5d49-4415-8df1-a3cdd6d1bf2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ab0eae69a074048a1374456fcfecbe0'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:43:07.155983 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-c37805b2-5d49-4415-8df1-a3cdd6d1bf2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ab0eae69a074048a1374456fcfecbe0'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:43:07.156674 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-c37805b2-5d49-4415-8df1-a3cdd6d1bf2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ab0eae69a074048a1374456fcfecbe0'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:43:07.227646 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-c37805b2-5d49-4415-8df1-a3cdd6d1bf2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ab0eae69a074048a1374456fcfecbe0'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:43:07.237942 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1310/2667] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:43:06 2022] DELETE /identity/v3/users/2ab0eae69a074048a1374456fcfecbe0 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:43:07.713079 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44e07d-414b-4391-8290-118aba98d815 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:43:07.714618 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-8b44e07d-414b-4391-8290-118aba98d815 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:43:07.745336 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44e07d-414b-4391-8290-118aba98d815 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:43:07.809247 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff13be9-334d-4571-952c-1ef1244b18c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:43:07.810400 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff13be9-334d-4571-952c-1ef1244b18c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:43:07.810912 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff13be9-334d-4571-952c-1ef1244b18c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:43:07.811466 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff13be9-334d-4571-952c-1ef1244b18c6 admin admin] PATH_INFO: `/v3/projects/f76ba82c003146419310d2fcadc0cd6e` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:43:07.812988 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff13be9-334d-4571-952c-1ef1244b18c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f76ba82c003146419310d2fcadc0cd6e)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:43:07.815575 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff13be9-334d-4571-952c-1ef1244b18c6 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:43:07.938676 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7ff13be9-334d-4571-952c-1ef1244b18c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f76ba82c003146419310d2fcadc0cd6e', 'request_id': 'req-7ff13be9-334d-4571-952c-1ef1244b18c6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:43:07.946834 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7ff13be9-334d-4571-952c-1ef1244b18c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f76ba82c003146419310d2fcadc0cd6e', 'request_id': 'req-7ff13be9-334d-4571-952c-1ef1244b18c6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:43:07.952876 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7ff13be9-334d-4571-952c-1ef1244b18c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f76ba82c003146419310d2fcadc0cd6e', 'request_id': 'req-7ff13be9-334d-4571-952c-1ef1244b18c6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:43:08.039428 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7ff13be9-334d-4571-952c-1ef1244b18c6 admin admin] The token cache is being invalidate because project f76ba82c003146419310d2fcadc0cd6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:43:08.039901 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7ff13be9-334d-4571-952c-1ef1244b18c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:43:08.040585 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-7ff13be9-334d-4571-952c-1ef1244b18c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:43:08.042150 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1358/2668] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:43:07 2022] DELETE /identity/v3/projects/f76ba82c003146419310d2fcadc0cd6e => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:43:08.047937 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8288ce4-19ae-4f96-8e3f-7c3fec70bb6e None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:43:08.048437 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8288ce4-19ae-4f96-8e3f-7c3fec70bb6e None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:43:08.048541 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8288ce4-19ae-4f96-8e3f-7c3fec70bb6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:43:08.096723 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-f8288ce4-19ae-4f96-8e3f-7c3fec70bb6e None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:43:08.097845 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-f8288ce4-19ae-4f96-8e3f-7c3fec70bb6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:43:08.135651 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:43:08.156508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1311/2669] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:43:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:43:08.213762 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d5594-7513-4968-90a8-0f175579589d None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:43:08.214055 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d5594-7513-4968-90a8-0f175579589d None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:43:08.214340 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7d5594-7513-4968-90a8-0f175579589d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:43:08.233060 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-7c7d5594-7513-4968-90a8-0f175579589d None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:43:08.234415 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-7c7d5594-7513-4968-90a8-0f175579589d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:43:08.242996 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1359/2670] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:43:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:43:08.247248 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8288ce4-19ae-4f96-8e3f-7c3fec70bb6e None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:43:08.257007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8288ce4-19ae-4f96-8e3f-7c3fec70bb6e None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:43:08.261546 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc38d153-0b1c-41f2-af69-5ae482ee0206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:43:08.262206 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38d153-0b1c-41f2-af69-5ae482ee0206 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:43:08.262492 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38d153-0b1c-41f2-af69-5ae482ee0206 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:43:08.262771 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc38d153-0b1c-41f2-af69-5ae482ee0206 admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:43:08.263548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc38d153-0b1c-41f2-af69-5ae482ee0206 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:43:08.263891 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc38d153-0b1c-41f2-af69-5ae482ee0206 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:43:08.265509 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc38d153-0b1c-41f2-af69-5ae482ee0206 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:43:08.277034 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1312/2671] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:43:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:43:08.287712 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d5594-7513-4968-90a8-0f175579589d None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:43:08.296868 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7d5594-7513-4968-90a8-0f175579589d None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:43:08.302163 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968bceed-c55c-45a3-bf7c-c95caba595b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:43:08.302876 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bceed-c55c-45a3-bf7c-c95caba595b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:43:08.303162 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bceed-c55c-45a3-bf7c-c95caba595b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:43:08.303442 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968bceed-c55c-45a3-bf7c-c95caba595b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:43:08.304161 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968bceed-c55c-45a3-bf7c-c95caba595b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:43:08.306056 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968bceed-c55c-45a3-bf7c-c95caba595b0 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:43:08.329023 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:43:08.427310 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1360/2672] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:43:08 2022] POST /identity/v3/projects => generated 365 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:43:08.431904 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc38d153-0b1c-41f2-af69-5ae482ee0206 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:43:08.442663 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc38d153-0b1c-41f2-af69-5ae482ee0206 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:43:08.501434 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa9607-9d06-4cd6-ba96-0a4da172b6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:43:08.502127 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa9607-9d06-4cd6-ba96-0a4da172b6fd admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:43:08.502413 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa9607-9d06-4cd6-ba96-0a4da172b6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:43:08.502693 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aa9607-9d06-4cd6-ba96-0a4da172b6fd admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:43:08.503457 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa9607-9d06-4cd6-ba96-0a4da172b6fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:43:08.505421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aa9607-9d06-4cd6-ba96-0a4da172b6fd admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:43:08.585045 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1313/2673] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:43:08 2022] POST /identity/v3/users => generated 283 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:43:08.589084 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968bceed-c55c-45a3-bf7c-c95caba595b0 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:43:08.599942 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968bceed-c55c-45a3-bf7c-c95caba595b0 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:43:08.604577 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d76d348-f444-42a7-9b33-1fc5f3994b5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:43:08.605230 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d76d348-f444-42a7-9b33-1fc5f3994b5b admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:43:08.605516 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d76d348-f444-42a7-9b33-1fc5f3994b5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:43:08.605795 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d76d348-f444-42a7-9b33-1fc5f3994b5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:43:08.606595 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d76d348-f444-42a7-9b33-1fc5f3994b5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:43:08.608227 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d76d348-f444-42a7-9b33-1fc5f3994b5b admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:43:08.618247 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1361/2674] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:43:08.621983 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa9607-9d06-4cd6-ba96-0a4da172b6fd admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:43:08.631834 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aa9607-9d06-4cd6-ba96-0a4da172b6fd admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:43:08.636435 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5a06cb-acac-4c8c-8106-76d37754c384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:43:08.637085 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a06cb-acac-4c8c-8106-76d37754c384 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:43:08.637372 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a06cb-acac-4c8c-8106-76d37754c384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:43:08.637652 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5a06cb-acac-4c8c-8106-76d37754c384 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:43:08.638377 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5a06cb-acac-4c8c-8106-76d37754c384 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:43:08.639997 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5a06cb-acac-4c8c-8106-76d37754c384 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:43:08.650156 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1314/2675] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:43:08 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:43:08.653894 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d76d348-f444-42a7-9b33-1fc5f3994b5b admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:43:08.664435 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d76d348-f444-42a7-9b33-1fc5f3994b5b admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:43:08.669055 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d48a2e-6f0a-44e5-8145-d84ce2d79b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:43:08.669835 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d48a2e-6f0a-44e5-8145-d84ce2d79b47 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:43:08.670117 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d48a2e-6f0a-44e5-8145-d84ce2d79b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:43:08.670387 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d48a2e-6f0a-44e5-8145-d84ce2d79b47 admin admin] PATH_INFO: `/v3/projects/08dfef82a5d74288a8cc05c93c605647/users/449547aaebea4e7593d8f2e10f1fdacd/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:43:08.688014 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d48a2e-6f0a-44e5-8145-d84ce2d79b47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08dfef82a5d74288a8cc05c93c605647, user_id=449547aaebea4e7593d8f2e10f1fdacd, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:43:08.691823 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d48a2e-6f0a-44e5-8145-d84ce2d79b47 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:43:08.779503 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1362/2676] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:43:08 2022] PUT /identity/v3/projects/08dfef82a5d74288a8cc05c93c605647/users/449547aaebea4e7593d8f2e10f1fdacd/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:43:08.792303 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577243e-3f8e-46ab-bad6-f2931be416cf None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:43:08.792649 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577243e-3f8e-46ab-bad6-f2931be416cf None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:43:08.792938 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0577243e-3f8e-46ab-bad6-f2931be416cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:43:08.819440 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-0577243e-3f8e-46ab-bad6-f2931be416cf None None] MFA Rules not processed for user `449547aaebea4e7593d8f2e10f1fdacd`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:43:08.820711 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-0577243e-3f8e-46ab-bad6-f2931be416cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:43:08.885006 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1315/2677] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:43:08 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:43:08.890025 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d48a2e-6f0a-44e5-8145-d84ce2d79b47 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:43:08.967368 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d48a2e-6f0a-44e5-8145-d84ce2d79b47 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:43:08.972486 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f33598-71b3-4591-a2f7-86876201c9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:43:08.973229 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f33598-71b3-4591-a2f7-86876201c9ed admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:43:08.973723 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f33598-71b3-4591-a2f7-86876201c9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:43:08.974188 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f33598-71b3-4591-a2f7-86876201c9ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:43:08.975114 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f33598-71b3-4591-a2f7-86876201c9ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:43:08.977533 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f33598-71b3-4591-a2f7-86876201c9ed admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:43:09.093186 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1363/2678] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:43:08 2022] POST /identity/v3/projects => generated 363 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:43:09.099136 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0577243e-3f8e-46ab-bad6-f2931be416cf None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:43:09.111932 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0577243e-3f8e-46ab-bad6-f2931be416cf None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:43:09.152310 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:43:09.177110 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51433c48-10e8-4478-bb2a-db94897f58e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:43:09.177889 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51433c48-10e8-4478-bb2a-db94897f58e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:43:09.178168 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51433c48-10e8-4478-bb2a-db94897f58e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:43:09.178444 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51433c48-10e8-4478-bb2a-db94897f58e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:43:09.179140 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51433c48-10e8-4478-bb2a-db94897f58e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:43:09.181089 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51433c48-10e8-4478-bb2a-db94897f58e8 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:43:09.276735 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1316/2679] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:43:09 2022] POST /identity/v3/users => generated 288 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:43:09.278008 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f33598-71b3-4591-a2f7-86876201c9ed admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:43:09.289348 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f33598-71b3-4591-a2f7-86876201c9ed admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:43:09.294604 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d29aeb-3b46-4948-9146-76ef730264a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:43:09.295356 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d29aeb-3b46-4948-9146-76ef730264a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:43:09.295761 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d29aeb-3b46-4948-9146-76ef730264a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:43:09.296095 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d29aeb-3b46-4948-9146-76ef730264a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:43:09.297174 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d29aeb-3b46-4948-9146-76ef730264a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:43:09.299353 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d29aeb-3b46-4948-9146-76ef730264a6 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:43:09.311142 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1364/2680] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:43:09.315909 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51433c48-10e8-4478-bb2a-db94897f58e8 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:43:09.325813 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51433c48-10e8-4478-bb2a-db94897f58e8 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:43:09.330403 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548873a7-ba5b-43a7-b611-dd0dd62d24a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:43:09.331276 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548873a7-ba5b-43a7-b611-dd0dd62d24a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:43:09.331557 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548873a7-ba5b-43a7-b611-dd0dd62d24a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:43:09.331867 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548873a7-ba5b-43a7-b611-dd0dd62d24a5 admin admin] PATH_INFO: `/v3/projects/7705b19c736d43fe8dd2339a1bf2be39/users/57dbc128cbef4246bf2226705454a584/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:43:09.346376 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548873a7-ba5b-43a7-b611-dd0dd62d24a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7705b19c736d43fe8dd2339a1bf2be39, user_id=57dbc128cbef4246bf2226705454a584, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:43:09.349068 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548873a7-ba5b-43a7-b611-dd0dd62d24a5 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:43:09.421069 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1317/2681] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:43:09 2022] PUT /identity/v3/projects/7705b19c736d43fe8dd2339a1bf2be39/users/57dbc128cbef4246bf2226705454a584/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:43:09.425680 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d29aeb-3b46-4948-9146-76ef730264a6 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:43:09.436382 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d29aeb-3b46-4948-9146-76ef730264a6 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:43:09.493386 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd1c208-bb1d-4f8e-9f82-1b959ba190b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:43:09.493986 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1c208-bb1d-4f8e-9f82-1b959ba190b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:43:09.494272 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1c208-bb1d-4f8e-9f82-1b959ba190b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:43:09.494578 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd1c208-bb1d-4f8e-9f82-1b959ba190b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:43:09.495307 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd1c208-bb1d-4f8e-9f82-1b959ba190b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:43:09.496946 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd1c208-bb1d-4f8e-9f82-1b959ba190b4 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:43:09.507423 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1365/2682] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:43:09 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:43:09.512685 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548873a7-ba5b-43a7-b611-dd0dd62d24a5 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:43:09.521637 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548873a7-ba5b-43a7-b611-dd0dd62d24a5 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:43:09.526105 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abad881-b0fd-4faa-b262-36fedf217ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:43:09.526974 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abad881-b0fd-4faa-b262-36fedf217ffc admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:43:09.527256 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abad881-b0fd-4faa-b262-36fedf217ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:43:09.527533 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abad881-b0fd-4faa-b262-36fedf217ffc admin admin] PATH_INFO: `/v3/projects/7705b19c736d43fe8dd2339a1bf2be39/users/57dbc128cbef4246bf2226705454a584/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:43:09.529492 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abad881-b0fd-4faa-b262-36fedf217ffc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7705b19c736d43fe8dd2339a1bf2be39, user_id=57dbc128cbef4246bf2226705454a584, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:43:09.532149 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abad881-b0fd-4faa-b262-36fedf217ffc admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:43:09.542380 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1318/2683] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:43:09 2022] PUT /identity/v3/projects/7705b19c736d43fe8dd2339a1bf2be39/users/57dbc128cbef4246bf2226705454a584/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:43:09.553640 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a339aeb-d7e5-4c51-ba64-b288203c69af None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:43:09.553936 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a339aeb-d7e5-4c51-ba64-b288203c69af None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:43:09.554215 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a339aeb-d7e5-4c51-ba64-b288203c69af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:43:09.579889 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-0a339aeb-d7e5-4c51-ba64-b288203c69af None None] MFA Rules not processed for user `57dbc128cbef4246bf2226705454a584`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:43:09.581119 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-0a339aeb-d7e5-4c51-ba64-b288203c69af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:43:09.598643 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:43:09.646729 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1366/2684] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:43:09 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:43:15.198648 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abad881-b0fd-4faa-b262-36fedf217ffc admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:43:15.200355 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-1abad881-b0fd-4faa-b262-36fedf217ffc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:43:15.267338 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abad881-b0fd-4faa-b262-36fedf217ffc admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:43:15.271410 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8646c1ab-dfb1-4557-b44e-88cbd3d83fad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:43:15.272079 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8646c1ab-dfb1-4557-b44e-88cbd3d83fad service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:43:15.272367 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8646c1ab-dfb1-4557-b44e-88cbd3d83fad service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:43:15.272648 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8646c1ab-dfb1-4557-b44e-88cbd3d83fad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:43:15.274373 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-8646c1ab-dfb1-4557-b44e-88cbd3d83fad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:43:15.342034 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8646c1ab-dfb1-4557-b44e-88cbd3d83fad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:43:15.343462 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8646c1ab-dfb1-4557-b44e-88cbd3d83fad service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:43:15.354274 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1319/2685] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:43:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:43:19.684428 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=20 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:43:25.350105 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=86,keystone:INSERT=5 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:44:52.383538 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a339aeb-d7e5-4c51-ba64-b288203c69af None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:44:52.385000 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-0a339aeb-d7e5-4c51-ba64-b288203c69af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:44:52.400695 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a339aeb-d7e5-4c51-ba64-b288203c69af None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:44:52.406064 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1150ba6-688b-44fb-bd3f-81d8c5957ffe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:44:52.406706 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1150ba6-688b-44fb-bd3f-81d8c5957ffe service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:44:52.406992 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1150ba6-688b-44fb-bd3f-81d8c5957ffe service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:44:52.407273 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1150ba6-688b-44fb-bd3f-81d8c5957ffe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:44:52.409515 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d1150ba6-688b-44fb-bd3f-81d8c5957ffe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:44:52.478807 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1150ba6-688b-44fb-bd3f-81d8c5957ffe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:44:52.480450 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1150ba6-688b-44fb-bd3f-81d8c5957ffe service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:44:52.490208 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1367/2686] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:44:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:44:52.554772 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8646c1ab-dfb1-4557-b44e-88cbd3d83fad service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:44:52.556237 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-8646c1ab-dfb1-4557-b44e-88cbd3d83fad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:44:52.566661 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8646c1ab-dfb1-4557-b44e-88cbd3d83fad service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:44:52.570343 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe8fa81-8196-470d-b34b-8e6fc0083133 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:44:52.570981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe8fa81-8196-470d-b34b-8e6fc0083133 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:44:52.571264 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe8fa81-8196-470d-b34b-8e6fc0083133 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:44:52.571540 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe8fa81-8196-470d-b34b-8e6fc0083133 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:44:52.573293 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-9fe8fa81-8196-470d-b34b-8e6fc0083133 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:44:52.586742 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe8fa81-8196-470d-b34b-8e6fc0083133 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:44:52.588229 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe8fa81-8196-470d-b34b-8e6fc0083133 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:44:52.600020 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1320/2687] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:44:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:45:02.485422 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=33 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:45:02.594453 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=6 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:45:05.129158 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1150ba6-688b-44fb-bd3f-81d8c5957ffe service nova] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:45:05.130863 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d1150ba6-688b-44fb-bd3f-81d8c5957ffe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:45:05.199463 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1150ba6-688b-44fb-bd3f-81d8c5957ffe service nova] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:45:05.203335 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1d213e-b649-45b3-a1fb-477a2c463be4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:45:05.204020 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d213e-b649-45b3-a1fb-477a2c463be4 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:05.204309 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d213e-b649-45b3-a1fb-477a2c463be4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:05.204592 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1d213e-b649-45b3-a1fb-477a2c463be4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:05.214243 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d213e-b649-45b3-a1fb-477a2c463be4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:45:05.215675 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1d213e-b649-45b3-a1fb-477a2c463be4 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:45:05.226386 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1368/2688] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:45:05 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:45:06.418196 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe8fa81-8196-470d-b34b-8e6fc0083133 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:45:06.487601 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe8fa81-8196-470d-b34b-8e6fc0083133 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:45:06.492723 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22613b42-d26a-4111-8b10-654c84eeb455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:45:06.493609 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22613b42-d26a-4111-8b10-654c84eeb455 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:06.493899 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22613b42-d26a-4111-8b10-654c84eeb455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:06.494177 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22613b42-d26a-4111-8b10-654c84eeb455 admin admin] PATH_INFO: `/v3/users/449547aaebea4e7593d8f2e10f1fdacd` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:06.495479 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22613b42-d26a-4111-8b10-654c84eeb455 admin admin] RBAC: Authorizing `identity:delete_user(user_id=449547aaebea4e7593d8f2e10f1fdacd)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:45:06.497453 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22613b42-d26a-4111-8b10-654c84eeb455 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:45:06.694052 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-22613b42-d26a-4111-8b10-654c84eeb455 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '449547aaebea4e7593d8f2e10f1fdacd'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:45:06.705705 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-22613b42-d26a-4111-8b10-654c84eeb455 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '449547aaebea4e7593d8f2e10f1fdacd'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:45:06.712510 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-22613b42-d26a-4111-8b10-654c84eeb455 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '449547aaebea4e7593d8f2e10f1fdacd'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:45:06.713167 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-22613b42-d26a-4111-8b10-654c84eeb455 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '449547aaebea4e7593d8f2e10f1fdacd'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:45:06.778947 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-22613b42-d26a-4111-8b10-654c84eeb455 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '449547aaebea4e7593d8f2e10f1fdacd'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:45:06.786085 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1321/2689] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:45:06 2022] DELETE /identity/v3/users/449547aaebea4e7593d8f2e10f1fdacd => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:45:06.790912 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941529de-e615-4440-bd3a-b8ec83c24fad None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:06.791406 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941529de-e615-4440-bd3a-b8ec83c24fad None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:06.791611 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941529de-e615-4440-bd3a-b8ec83c24fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:06.810064 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-941529de-e615-4440-bd3a-b8ec83c24fad None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:45:06.811317 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-941529de-e615-4440-bd3a-b8ec83c24fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:45:06.869705 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1369/2690] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:45:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:45:06.879348 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22613b42-d26a-4111-8b10-654c84eeb455 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:45:06.881025 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-22613b42-d26a-4111-8b10-654c84eeb455 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:45:06.936678 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22613b42-d26a-4111-8b10-654c84eeb455 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:45:06.940317 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252d971d-e55d-43ab-a9b5-dce45c1b658d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:45:06.940950 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d971d-e55d-43ab-a9b5-dce45c1b658d service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:06.941239 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d971d-e55d-43ab-a9b5-dce45c1b658d service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:06.941504 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d971d-e55d-43ab-a9b5-dce45c1b658d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:06.950840 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252d971d-e55d-43ab-a9b5-dce45c1b658d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:45:06.952266 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252d971d-e55d-43ab-a9b5-dce45c1b658d service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:45:06.962448 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1322/2691] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:45:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:45:07.412649 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941529de-e615-4440-bd3a-b8ec83c24fad None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:45:07.415559 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-941529de-e615-4440-bd3a-b8ec83c24fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:45:07.427356 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941529de-e615-4440-bd3a-b8ec83c24fad None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:45:07.432138 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa6e14-ac1f-4ee2-a79e-76a39d4c2d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:45:07.433136 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa6e14-ac1f-4ee2-a79e-76a39d4c2d15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:07.433422 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa6e14-ac1f-4ee2-a79e-76a39d4c2d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:07.433705 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fa6e14-ac1f-4ee2-a79e-76a39d4c2d15 admin admin] PATH_INFO: `/v3/projects/08dfef82a5d74288a8cc05c93c605647` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:07.434939 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa6e14-ac1f-4ee2-a79e-76a39d4c2d15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08dfef82a5d74288a8cc05c93c605647)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:45:07.436898 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fa6e14-ac1f-4ee2-a79e-76a39d4c2d15 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:45:07.559446 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-24fa6e14-ac1f-4ee2-a79e-76a39d4c2d15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08dfef82a5d74288a8cc05c93c605647', 'request_id': 'req-24fa6e14-ac1f-4ee2-a79e-76a39d4c2d15'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:45:07.566119 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-24fa6e14-ac1f-4ee2-a79e-76a39d4c2d15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08dfef82a5d74288a8cc05c93c605647', 'request_id': 'req-24fa6e14-ac1f-4ee2-a79e-76a39d4c2d15'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:45:07.571713 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-24fa6e14-ac1f-4ee2-a79e-76a39d4c2d15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08dfef82a5d74288a8cc05c93c605647', 'request_id': 'req-24fa6e14-ac1f-4ee2-a79e-76a39d4c2d15'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:45:07.629125 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-24fa6e14-ac1f-4ee2-a79e-76a39d4c2d15 admin admin] The token cache is being invalidate because project 08dfef82a5d74288a8cc05c93c605647 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:45:07.629418 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-24fa6e14-ac1f-4ee2-a79e-76a39d4c2d15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:45:07.630041 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-24fa6e14-ac1f-4ee2-a79e-76a39d4c2d15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:45:07.631584 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1370/2692] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:45:07 2022] DELETE /identity/v3/projects/08dfef82a5d74288a8cc05c93c605647 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:45:07.635624 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252d971d-e55d-43ab-a9b5-dce45c1b658d service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:45:07.645931 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252d971d-e55d-43ab-a9b5-dce45c1b658d service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:45:07.664326 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:45:07.701953 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa62f7f-7c4c-42f4-b57e-304ac7da3169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:45:07.702818 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa62f7f-7c4c-42f4-b57e-304ac7da3169 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:07.703106 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa62f7f-7c4c-42f4-b57e-304ac7da3169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:07.703389 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa62f7f-7c4c-42f4-b57e-304ac7da3169 admin admin] PATH_INFO: `/v3/users/57dbc128cbef4246bf2226705454a584` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:07.704621 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa62f7f-7c4c-42f4-b57e-304ac7da3169 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57dbc128cbef4246bf2226705454a584)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:45:07.706674 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa62f7f-7c4c-42f4-b57e-304ac7da3169 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:45:07.904411 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6aa62f7f-7c4c-42f4-b57e-304ac7da3169 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57dbc128cbef4246bf2226705454a584'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:45:07.917192 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6aa62f7f-7c4c-42f4-b57e-304ac7da3169 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57dbc128cbef4246bf2226705454a584'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:45:07.924260 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6aa62f7f-7c4c-42f4-b57e-304ac7da3169 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57dbc128cbef4246bf2226705454a584'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:45:07.924943 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6aa62f7f-7c4c-42f4-b57e-304ac7da3169 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57dbc128cbef4246bf2226705454a584'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:45:07.989939 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6aa62f7f-7c4c-42f4-b57e-304ac7da3169 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57dbc128cbef4246bf2226705454a584'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:45:07.996009 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1323/2693] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:45:07 2022] DELETE /identity/v3/users/57dbc128cbef4246bf2226705454a584 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:45:08.390881 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa6e14-ac1f-4ee2-a79e-76a39d4c2d15 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:45:08.392381 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-24fa6e14-ac1f-4ee2-a79e-76a39d4c2d15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:45:08.406997 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fa6e14-ac1f-4ee2-a79e-76a39d4c2d15 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:45:08.419999 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:45:08.465954 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265baed3-380f-401c-9f5b-e604d95b7724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:45:08.466834 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265baed3-380f-401c-9f5b-e604d95b7724 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:08.467118 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265baed3-380f-401c-9f5b-e604d95b7724 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:08.467390 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265baed3-380f-401c-9f5b-e604d95b7724 admin admin] PATH_INFO: `/v3/projects/7705b19c736d43fe8dd2339a1bf2be39` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:08.468703 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265baed3-380f-401c-9f5b-e604d95b7724 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7705b19c736d43fe8dd2339a1bf2be39)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:45:08.470883 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265baed3-380f-401c-9f5b-e604d95b7724 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:45:08.710138 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-265baed3-380f-401c-9f5b-e604d95b7724 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7705b19c736d43fe8dd2339a1bf2be39', 'request_id': 'req-265baed3-380f-401c-9f5b-e604d95b7724'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:45:08.716600 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-265baed3-380f-401c-9f5b-e604d95b7724 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7705b19c736d43fe8dd2339a1bf2be39', 'request_id': 'req-265baed3-380f-401c-9f5b-e604d95b7724'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:45:08.723035 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-265baed3-380f-401c-9f5b-e604d95b7724 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7705b19c736d43fe8dd2339a1bf2be39', 'request_id': 'req-265baed3-380f-401c-9f5b-e604d95b7724'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:45:08.806378 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-265baed3-380f-401c-9f5b-e604d95b7724 admin admin] The token cache is being invalidate because project 7705b19c736d43fe8dd2339a1bf2be39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:45:08.806670 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-265baed3-380f-401c-9f5b-e604d95b7724 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:45:08.807325 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-265baed3-380f-401c-9f5b-e604d95b7724 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:45:08.808945 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1371/2694] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:45:08 2022] DELETE /identity/v3/projects/7705b19c736d43fe8dd2339a1bf2be39 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:45:08.813977 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1dc142-97a2-480a-9e76-8058cf23fcc3 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:08.816749 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1dc142-97a2-480a-9e76-8058cf23fcc3 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:08.816749 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1dc142-97a2-480a-9e76-8058cf23fcc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:08.833785 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-3b1dc142-97a2-480a-9e76-8058cf23fcc3 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:45:08.835030 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-3b1dc142-97a2-480a-9e76-8058cf23fcc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:45:08.891914 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1324/2695] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:45:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:45:08.900383 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa69ce-f7c0-406c-b659-45f708a65285 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:08.900677 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa69ce-f7c0-406c-b659-45f708a65285 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:08.900955 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfa69ce-f7c0-406c-b659-45f708a65285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:08.918718 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-1cfa69ce-f7c0-406c-b659-45f708a65285 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:45:08.919954 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-1cfa69ce-f7c0-406c-b659-45f708a65285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:45:08.927909 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1372/2696] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:45:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:45:08.931932 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1dc142-97a2-480a-9e76-8058cf23fcc3 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:45:08.942381 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1dc142-97a2-480a-9e76-8058cf23fcc3 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:45:08.947277 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe780f9-31c2-48f2-8f6e-04e34d87d5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:45:08.947969 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe780f9-31c2-48f2-8f6e-04e34d87d5a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:08.948259 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe780f9-31c2-48f2-8f6e-04e34d87d5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:08.948538 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe780f9-31c2-48f2-8f6e-04e34d87d5a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:08.949264 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe780f9-31c2-48f2-8f6e-04e34d87d5a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:45:08.949574 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe780f9-31c2-48f2-8f6e-04e34d87d5a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:45:08.951163 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe780f9-31c2-48f2-8f6e-04e34d87d5a4 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:45:08.963760 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1325/2697] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:45:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:45:08.968125 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa69ce-f7c0-406c-b659-45f708a65285 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:45:08.979673 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfa69ce-f7c0-406c-b659-45f708a65285 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:45:08.984385 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7062519-642b-412c-abb4-cbcd636957f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:45:08.985036 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7062519-642b-412c-abb4-cbcd636957f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:08.985322 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7062519-642b-412c-abb4-cbcd636957f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:08.985601 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7062519-642b-412c-abb4-cbcd636957f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:08.986362 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7062519-642b-412c-abb4-cbcd636957f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:45:08.988271 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7062519-642b-412c-abb4-cbcd636957f8 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:45:09.094001 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1373/2698] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:45:08 2022] POST /identity/v3/projects => generated 365 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:45:09.098173 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe780f9-31c2-48f2-8f6e-04e34d87d5a4 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:45:09.108058 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe780f9-31c2-48f2-8f6e-04e34d87d5a4 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:45:09.133339 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:45:09.164581 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ec5fa-3b27-4642-92bf-191dd13ab6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:45:09.165254 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ec5fa-3b27-4642-92bf-191dd13ab6eb admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:09.165544 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ec5fa-3b27-4642-92bf-191dd13ab6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:09.165823 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65ec5fa-3b27-4642-92bf-191dd13ab6eb admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:09.166523 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65ec5fa-3b27-4642-92bf-191dd13ab6eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:45:09.168481 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65ec5fa-3b27-4642-92bf-191dd13ab6eb admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:45:09.245082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1326/2699] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:45:09 2022] POST /identity/v3/users => generated 283 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:45:09.249162 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7062519-642b-412c-abb4-cbcd636957f8 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:45:09.260258 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7062519-642b-412c-abb4-cbcd636957f8 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:45:09.264885 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5100e65b-2371-46b7-b59f-a344008d2660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:45:09.265543 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5100e65b-2371-46b7-b59f-a344008d2660 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:09.265828 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5100e65b-2371-46b7-b59f-a344008d2660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:09.266103 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5100e65b-2371-46b7-b59f-a344008d2660 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:09.266832 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5100e65b-2371-46b7-b59f-a344008d2660 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:45:09.268492 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5100e65b-2371-46b7-b59f-a344008d2660 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:45:09.278836 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1374/2700] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:45:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:45:09.286324 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ec5fa-3b27-4642-92bf-191dd13ab6eb admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:45:09.296760 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65ec5fa-3b27-4642-92bf-191dd13ab6eb admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:45:09.301950 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d93d7-f3b2-4de8-9764-7168a699bbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:45:09.302610 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d93d7-f3b2-4de8-9764-7168a699bbdb admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:09.302945 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d93d7-f3b2-4de8-9764-7168a699bbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:09.303228 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d93d7-f3b2-4de8-9764-7168a699bbdb admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:09.303986 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316d93d7-f3b2-4de8-9764-7168a699bbdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:45:09.305571 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316d93d7-f3b2-4de8-9764-7168a699bbdb admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:45:09.315593 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1327/2701] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:45:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:45:09.320516 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5100e65b-2371-46b7-b59f-a344008d2660 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:45:09.332576 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5100e65b-2371-46b7-b59f-a344008d2660 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:45:09.337195 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4012f291-2543-4f2c-a4ef-2533f1ccb66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:45:09.338116 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4012f291-2543-4f2c-a4ef-2533f1ccb66a admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:09.338396 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4012f291-2543-4f2c-a4ef-2533f1ccb66a admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:09.338665 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4012f291-2543-4f2c-a4ef-2533f1ccb66a admin admin] PATH_INFO: `/v3/projects/9d4f1b68655f45c0aa862b309a5c88d4/users/027b47fa8e7b4e148c29f07a91205dad/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:09.352876 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4012f291-2543-4f2c-a4ef-2533f1ccb66a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d4f1b68655f45c0aa862b309a5c88d4, user_id=027b47fa8e7b4e148c29f07a91205dad, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:45:09.355573 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4012f291-2543-4f2c-a4ef-2533f1ccb66a admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:45:09.453180 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1375/2702] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:45:09 2022] PUT /identity/v3/projects/9d4f1b68655f45c0aa862b309a5c88d4/users/027b47fa8e7b4e148c29f07a91205dad/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:45:09.463602 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd427df-5f3b-49ad-95a5-db5c9e5d8394 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:09.463923 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd427df-5f3b-49ad-95a5-db5c9e5d8394 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:09.464205 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd427df-5f3b-49ad-95a5-db5c9e5d8394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:09.496064 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-9dd427df-5f3b-49ad-95a5-db5c9e5d8394 None None] MFA Rules not processed for user `027b47fa8e7b4e148c29f07a91205dad`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:45:09.496064 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-9dd427df-5f3b-49ad-95a5-db5c9e5d8394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:45:09.588371 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1328/2703] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:45:09 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:45:09.592711 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4012f291-2543-4f2c-a4ef-2533f1ccb66a admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:45:09.640150 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:45:09.657070 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4012f291-2543-4f2c-a4ef-2533f1ccb66a admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:45:09.662565 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6d921f-1742-42e7-801d-89150c147718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:45:09.663244 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6d921f-1742-42e7-801d-89150c147718 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:09.663528 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6d921f-1742-42e7-801d-89150c147718 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:09.663843 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6d921f-1742-42e7-801d-89150c147718 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:09.664540 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6d921f-1742-42e7-801d-89150c147718 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:45:09.666461 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6d921f-1742-42e7-801d-89150c147718 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:45:09.756300 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1376/2704] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:45:09 2022] POST /identity/v3/projects => generated 365 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:45:09.760203 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd427df-5f3b-49ad-95a5-db5c9e5d8394 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:45:09.773786 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd427df-5f3b-49ad-95a5-db5c9e5d8394 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:45:09.832546 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a55454-9853-4e36-a65e-ed4e2e9565ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:45:09.833273 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a55454-9853-4e36-a65e-ed4e2e9565ad admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:09.833497 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a55454-9853-4e36-a65e-ed4e2e9565ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:09.833888 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a55454-9853-4e36-a65e-ed4e2e9565ad admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:09.834619 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a55454-9853-4e36-a65e-ed4e2e9565ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:45:09.836601 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a55454-9853-4e36-a65e-ed4e2e9565ad admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:45:09.930547 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1329/2705] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:45:09 2022] POST /identity/v3/users => generated 289 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:45:09.934619 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6d921f-1742-42e7-801d-89150c147718 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:45:09.945006 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6d921f-1742-42e7-801d-89150c147718 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:45:09.949802 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c851e-668e-47f1-9153-58a5f50fda2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:45:09.950463 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c851e-668e-47f1-9153-58a5f50fda2b admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:09.950743 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c851e-668e-47f1-9153-58a5f50fda2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:09.951018 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c851e-668e-47f1-9153-58a5f50fda2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:09.951844 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983c851e-668e-47f1-9153-58a5f50fda2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:45:09.953416 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983c851e-668e-47f1-9153-58a5f50fda2b admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:45:09.963655 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1377/2706] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:45:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:45:09.967566 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a55454-9853-4e36-a65e-ed4e2e9565ad admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:45:09.978107 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a55454-9853-4e36-a65e-ed4e2e9565ad admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:45:09.982641 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942fa74-f3ef-4a88-9d06-137353cacb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:45:09.983501 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942fa74-f3ef-4a88-9d06-137353cacb95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:09.983810 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942fa74-f3ef-4a88-9d06-137353cacb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:09.984091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2942fa74-f3ef-4a88-9d06-137353cacb95 admin admin] PATH_INFO: `/v3/projects/e5ad15a6072143e092fe7c6bac1b6544/users/c7af323e12064c61a6987a55213bff21/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:09.998817 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2942fa74-f3ef-4a88-9d06-137353cacb95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5ad15a6072143e092fe7c6bac1b6544, user_id=c7af323e12064c61a6987a55213bff21, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:45:10.001474 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2942fa74-f3ef-4a88-9d06-137353cacb95 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:45:10.075388 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1330/2707] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:45:09 2022] PUT /identity/v3/projects/e5ad15a6072143e092fe7c6bac1b6544/users/c7af323e12064c61a6987a55213bff21/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:45:10.079178 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c851e-668e-47f1-9153-58a5f50fda2b admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:45:10.089758 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c851e-668e-47f1-9153-58a5f50fda2b admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:45:10.155266 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb05cad6-4d50-454d-ba89-89153841e691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:45:10.155967 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05cad6-4d50-454d-ba89-89153841e691 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:10.156256 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05cad6-4d50-454d-ba89-89153841e691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:10.156541 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05cad6-4d50-454d-ba89-89153841e691 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:10.157260 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb05cad6-4d50-454d-ba89-89153841e691 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:45:10.158844 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb05cad6-4d50-454d-ba89-89153841e691 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:45:10.169898 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1378/2708] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:45:10 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:45:10.174165 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942fa74-f3ef-4a88-9d06-137353cacb95 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:45:10.179607 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:45:10.184916 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2942fa74-f3ef-4a88-9d06-137353cacb95 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:45:10.189420 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc67e33a-6f93-4299-91e0-eb758db19068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:45:10.190291 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67e33a-6f93-4299-91e0-eb758db19068 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:10.190574 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67e33a-6f93-4299-91e0-eb758db19068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:10.190849 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc67e33a-6f93-4299-91e0-eb758db19068 admin admin] PATH_INFO: `/v3/projects/e5ad15a6072143e092fe7c6bac1b6544/users/c7af323e12064c61a6987a55213bff21/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:10.192823 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc67e33a-6f93-4299-91e0-eb758db19068 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5ad15a6072143e092fe7c6bac1b6544, user_id=c7af323e12064c61a6987a55213bff21, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:45:10.195481 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc67e33a-6f93-4299-91e0-eb758db19068 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:45:10.205293 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1331/2709] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:45:10 2022] PUT /identity/v3/projects/e5ad15a6072143e092fe7c6bac1b6544/users/c7af323e12064c61a6987a55213bff21/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:45:10.216810 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5629a11-526b-46d1-bbf7-e3d571c4a99a None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:10.217105 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5629a11-526b-46d1-bbf7-e3d571c4a99a None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:10.217386 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5629a11-526b-46d1-bbf7-e3d571c4a99a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:10.243569 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-b5629a11-526b-46d1-bbf7-e3d571c4a99a None None] MFA Rules not processed for user `c7af323e12064c61a6987a55213bff21`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:45:10.244807 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b5629a11-526b-46d1-bbf7-e3d571c4a99a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:45:10.310596 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1379/2710] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:45:10 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:45:10.322043 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc67e33a-6f93-4299-91e0-eb758db19068 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:45:10.323717 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-bc67e33a-6f93-4299-91e0-eb758db19068 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:45:10.379660 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc67e33a-6f93-4299-91e0-eb758db19068 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:45:10.383270 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a311b-a00c-4a18-a588-9b82913497b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:45:10.383936 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a311b-a00c-4a18-a588-9b82913497b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:10.384220 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a311b-a00c-4a18-a588-9b82913497b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:10.384496 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a311b-a00c-4a18-a588-9b82913497b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:10.393229 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787a311b-a00c-4a18-a588-9b82913497b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:45:10.395312 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787a311b-a00c-4a18-a588-9b82913497b4 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:45:10.404789 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1332/2711] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:45:10 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:45:10.648732 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5629a11-526b-46d1-bbf7-e3d571c4a99a None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:45:10.658559 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5629a11-526b-46d1-bbf7-e3d571c4a99a None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:45:10.662571 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c92781-bf0d-4f63-8da9-8ea06bf72cb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:45:10.663257 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c92781-bf0d-4f63-8da9-8ea06bf72cb1 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:10.663536 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c92781-bf0d-4f63-8da9-8ea06bf72cb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:10.663846 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c92781-bf0d-4f63-8da9-8ea06bf72cb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:10.698954 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:45:10.722445 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c92781-bf0d-4f63-8da9-8ea06bf72cb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:45:10.723910 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c92781-bf0d-4f63-8da9-8ea06bf72cb1 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:45:10.732756 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1380/2712] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:45:10 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:45:14.266701 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a311b-a00c-4a18-a588-9b82913497b4 service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:45:14.329433 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a311b-a00c-4a18-a588-9b82913497b4 service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:45:14.334617 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d7c705-2367-49c3-81a5-edc368f8fb97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:45:14.335495 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7c705-2367-49c3-81a5-edc368f8fb97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:14.335841 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7c705-2367-49c3-81a5-edc368f8fb97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:14.336134 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d7c705-2367-49c3-81a5-edc368f8fb97 admin admin] PATH_INFO: `/v3/users/027b47fa8e7b4e148c29f07a91205dad` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:14.337316 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d7c705-2367-49c3-81a5-edc368f8fb97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=027b47fa8e7b4e148c29f07a91205dad)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:45:14.339234 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d7c705-2367-49c3-81a5-edc368f8fb97 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:45:14.558014 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-c0d7c705-2367-49c3-81a5-edc368f8fb97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '027b47fa8e7b4e148c29f07a91205dad'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:45:14.570445 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-c0d7c705-2367-49c3-81a5-edc368f8fb97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '027b47fa8e7b4e148c29f07a91205dad'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:45:14.577130 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-c0d7c705-2367-49c3-81a5-edc368f8fb97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '027b47fa8e7b4e148c29f07a91205dad'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:45:14.577795 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-c0d7c705-2367-49c3-81a5-edc368f8fb97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '027b47fa8e7b4e148c29f07a91205dad'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:45:14.646419 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-c0d7c705-2367-49c3-81a5-edc368f8fb97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '027b47fa8e7b4e148c29f07a91205dad'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:45:14.652332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1333/2713] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:45:14 2022] DELETE /identity/v3/users/027b47fa8e7b4e148c29f07a91205dad => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:45:14.657138 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6578326-5c0e-4b81-8737-f52460f6ef75 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:14.657745 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6578326-5c0e-4b81-8737-f52460f6ef75 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:14.657745 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6578326-5c0e-4b81-8737-f52460f6ef75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:14.674854 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-f6578326-5c0e-4b81-8737-f52460f6ef75 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:45:14.676608 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f6578326-5c0e-4b81-8737-f52460f6ef75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:45:14.738785 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1381/2714] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:45:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:45:14.748818 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d7c705-2367-49c3-81a5-edc368f8fb97 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:45:14.750288 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c0d7c705-2367-49c3-81a5-edc368f8fb97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:45:14.764955 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:45:14.808873 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d7c705-2367-49c3-81a5-edc368f8fb97 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:45:14.812531 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1253b-a4c9-4fa9-a9b5-49b659aabdfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:45:14.813184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1253b-a4c9-4fa9-a9b5-49b659aabdfd service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:14.813478 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1253b-a4c9-4fa9-a9b5-49b659aabdfd service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:14.813759 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b1253b-a4c9-4fa9-a9b5-49b659aabdfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:14.823881 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b1253b-a4c9-4fa9-a9b5-49b659aabdfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:45:14.826486 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b1253b-a4c9-4fa9-a9b5-49b659aabdfd service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:45:14.834607 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1334/2715] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:45:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:45:15.277344 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6578326-5c0e-4b81-8737-f52460f6ef75 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:45:15.278818 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f6578326-5c0e-4b81-8737-f52460f6ef75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:45:15.293297 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6578326-5c0e-4b81-8737-f52460f6ef75 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:45:15.298540 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b565a0-dd80-49a5-b1ca-918c8bf40349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:45:15.299475 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b565a0-dd80-49a5-b1ca-918c8bf40349 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:15.299874 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b565a0-dd80-49a5-b1ca-918c8bf40349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:15.300233 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b565a0-dd80-49a5-b1ca-918c8bf40349 admin admin] PATH_INFO: `/v3/projects/9d4f1b68655f45c0aa862b309a5c88d4` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:15.301536 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b565a0-dd80-49a5-b1ca-918c8bf40349 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d4f1b68655f45c0aa862b309a5c88d4)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:45:15.303564 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b565a0-dd80-49a5-b1ca-918c8bf40349 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:45:15.407541 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-18b565a0-dd80-49a5-b1ca-918c8bf40349 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d4f1b68655f45c0aa862b309a5c88d4', 'request_id': 'req-18b565a0-dd80-49a5-b1ca-918c8bf40349'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:45:15.414531 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-18b565a0-dd80-49a5-b1ca-918c8bf40349 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d4f1b68655f45c0aa862b309a5c88d4', 'request_id': 'req-18b565a0-dd80-49a5-b1ca-918c8bf40349'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:45:15.420156 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-18b565a0-dd80-49a5-b1ca-918c8bf40349 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d4f1b68655f45c0aa862b309a5c88d4', 'request_id': 'req-18b565a0-dd80-49a5-b1ca-918c8bf40349'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:45:15.497467 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-18b565a0-dd80-49a5-b1ca-918c8bf40349 admin admin] The token cache is being invalidate because project 9d4f1b68655f45c0aa862b309a5c88d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:45:15.497760 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-18b565a0-dd80-49a5-b1ca-918c8bf40349 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:45:15.498448 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-18b565a0-dd80-49a5-b1ca-918c8bf40349 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:45:15.500150 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1382/2716] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:45:15 2022] DELETE /identity/v3/projects/9d4f1b68655f45c0aa862b309a5c88d4 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:45:15.504746 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1253b-a4c9-4fa9-a9b5-49b659aabdfd service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:45:15.514312 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b1253b-a4c9-4fa9-a9b5-49b659aabdfd service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:45:15.574665 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4b2119-675c-4643-af3f-97932f16010e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:45:15.575549 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b2119-675c-4643-af3f-97932f16010e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:15.575875 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b2119-675c-4643-af3f-97932f16010e admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:15.576162 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4b2119-675c-4643-af3f-97932f16010e admin admin] PATH_INFO: `/v3/users/c7af323e12064c61a6987a55213bff21` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:15.577348 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4b2119-675c-4643-af3f-97932f16010e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7af323e12064c61a6987a55213bff21)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:45:15.579279 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4b2119-675c-4643-af3f-97932f16010e admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:45:15.779209 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ca4b2119-675c-4643-af3f-97932f16010e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7af323e12064c61a6987a55213bff21'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:45:15.792567 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ca4b2119-675c-4643-af3f-97932f16010e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7af323e12064c61a6987a55213bff21'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:45:15.799669 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ca4b2119-675c-4643-af3f-97932f16010e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7af323e12064c61a6987a55213bff21'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:45:15.801050 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ca4b2119-675c-4643-af3f-97932f16010e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7af323e12064c61a6987a55213bff21'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:45:15.867848 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-ca4b2119-675c-4643-af3f-97932f16010e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7af323e12064c61a6987a55213bff21'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:45:15.874512 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1335/2717] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:45:15 2022] DELETE /identity/v3/users/c7af323e12064c61a6987a55213bff21 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:45:16.303226 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b565a0-dd80-49a5-b1ca-918c8bf40349 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:45:16.304802 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-18b565a0-dd80-49a5-b1ca-918c8bf40349 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:45:16.318751 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b565a0-dd80-49a5-b1ca-918c8bf40349 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:45:16.377028 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71060909-88a0-44e2-9657-fcac9f3370a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:45:16.377946 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71060909-88a0-44e2-9657-fcac9f3370a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:16.379205 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71060909-88a0-44e2-9657-fcac9f3370a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:16.379489 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71060909-88a0-44e2-9657-fcac9f3370a6 admin admin] PATH_INFO: `/v3/projects/e5ad15a6072143e092fe7c6bac1b6544` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:16.380724 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71060909-88a0-44e2-9657-fcac9f3370a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5ad15a6072143e092fe7c6bac1b6544)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:45:16.384208 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71060909-88a0-44e2-9657-fcac9f3370a6 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:45:16.384208 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:45:16.499067 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-71060909-88a0-44e2-9657-fcac9f3370a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5ad15a6072143e092fe7c6bac1b6544', 'request_id': 'req-71060909-88a0-44e2-9657-fcac9f3370a6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:45:16.505681 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-71060909-88a0-44e2-9657-fcac9f3370a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5ad15a6072143e092fe7c6bac1b6544', 'request_id': 'req-71060909-88a0-44e2-9657-fcac9f3370a6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:45:16.511038 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-71060909-88a0-44e2-9657-fcac9f3370a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5ad15a6072143e092fe7c6bac1b6544', 'request_id': 'req-71060909-88a0-44e2-9657-fcac9f3370a6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:45:16.577048 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-71060909-88a0-44e2-9657-fcac9f3370a6 admin admin] The token cache is being invalidate because project e5ad15a6072143e092fe7c6bac1b6544 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:45:16.577334 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-71060909-88a0-44e2-9657-fcac9f3370a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:45:16.577949 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-71060909-88a0-44e2-9657-fcac9f3370a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:45:16.579564 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1383/2718] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:45:16 2022] DELETE /identity/v3/projects/e5ad15a6072143e092fe7c6bac1b6544 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:45:16.592613 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b7fd1-14c8-4a30-9818-5d40f3800646 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:16.593107 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b7fd1-14c8-4a30-9818-5d40f3800646 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:16.593226 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1b7fd1-14c8-4a30-9818-5d40f3800646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:16.611167 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-9e1b7fd1-14c8-4a30-9818-5d40f3800646 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:45:16.613607 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-9e1b7fd1-14c8-4a30-9818-5d40f3800646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:45:16.640975 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:45:16.670424 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1336/2719] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:45:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:45:16.678639 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b3542-22aa-4ad0-9162-5a5470f7009b None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:16.678935 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b3542-22aa-4ad0-9162-5a5470f7009b None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:16.679215 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b3542-22aa-4ad0-9162-5a5470f7009b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:16.697855 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-9e2b3542-22aa-4ad0-9162-5a5470f7009b None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:45:16.704437 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-9e2b3542-22aa-4ad0-9162-5a5470f7009b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:45:16.706687 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1384/2720] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:45:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:45:16.710817 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1b7fd1-14c8-4a30-9818-5d40f3800646 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:45:16.721441 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1b7fd1-14c8-4a30-9818-5d40f3800646 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:45:16.727473 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60278e55-ba37-420c-9e23-afbc93c99450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:45:16.727473 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60278e55-ba37-420c-9e23-afbc93c99450 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:16.727473 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60278e55-ba37-420c-9e23-afbc93c99450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:16.727473 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60278e55-ba37-420c-9e23-afbc93c99450 admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:16.728004 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60278e55-ba37-420c-9e23-afbc93c99450 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:45:16.728235 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60278e55-ba37-420c-9e23-afbc93c99450 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:45:16.729839 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60278e55-ba37-420c-9e23-afbc93c99450 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:45:16.741964 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1337/2721] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:45:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:45:16.746220 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b3542-22aa-4ad0-9162-5a5470f7009b None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:45:16.755920 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2b3542-22aa-4ad0-9162-5a5470f7009b None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:45:16.760451 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a658f-1126-452f-b2fb-d51bf04062b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:45:16.761093 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a658f-1126-452f-b2fb-d51bf04062b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:16.761374 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a658f-1126-452f-b2fb-d51bf04062b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:16.761642 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a658f-1126-452f-b2fb-d51bf04062b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:16.762324 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7a658f-1126-452f-b2fb-d51bf04062b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:45:16.764258 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7a658f-1126-452f-b2fb-d51bf04062b5 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:45:16.832095 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1385/2722] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:45:16 2022] POST /identity/v3/projects => generated 371 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:45:16.836124 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60278e55-ba37-420c-9e23-afbc93c99450 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:45:16.846204 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60278e55-ba37-420c-9e23-afbc93c99450 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:45:16.903609 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36194443-ddff-49a5-9f95-adaee9cf0e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:45:16.904316 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36194443-ddff-49a5-9f95-adaee9cf0e2d admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:16.904603 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36194443-ddff-49a5-9f95-adaee9cf0e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:16.904880 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36194443-ddff-49a5-9f95-adaee9cf0e2d admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:16.905616 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36194443-ddff-49a5-9f95-adaee9cf0e2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:45:16.907510 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36194443-ddff-49a5-9f95-adaee9cf0e2d admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:45:17.010749 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1338/2723] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:45:16 2022] POST /identity/v3/users => generated 286 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:45:17.014762 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a658f-1126-452f-b2fb-d51bf04062b5 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:45:17.025032 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a658f-1126-452f-b2fb-d51bf04062b5 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:45:17.029673 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160be5c0-20ed-47ed-8f52-3d4a84f4af6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:45:17.030329 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160be5c0-20ed-47ed-8f52-3d4a84f4af6b admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:17.030617 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160be5c0-20ed-47ed-8f52-3d4a84f4af6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:17.030896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160be5c0-20ed-47ed-8f52-3d4a84f4af6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:17.031619 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160be5c0-20ed-47ed-8f52-3d4a84f4af6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:45:17.033246 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160be5c0-20ed-47ed-8f52-3d4a84f4af6b admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:45:17.046795 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1386/2724] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:45:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:45:17.051930 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36194443-ddff-49a5-9f95-adaee9cf0e2d admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:45:17.062158 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36194443-ddff-49a5-9f95-adaee9cf0e2d admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:45:17.066774 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec8fc2-6272-4181-ae8a-f5ba95a4af24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:45:17.067453 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec8fc2-6272-4181-ae8a-f5ba95a4af24 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:17.067749 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec8fc2-6272-4181-ae8a-f5ba95a4af24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:17.068041 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec8fc2-6272-4181-ae8a-f5ba95a4af24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:17.068766 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec8fc2-6272-4181-ae8a-f5ba95a4af24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:45:17.070337 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec8fc2-6272-4181-ae8a-f5ba95a4af24 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:45:17.080215 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1339/2725] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:45:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:45:17.084416 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160be5c0-20ed-47ed-8f52-3d4a84f4af6b admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:45:17.094232 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160be5c0-20ed-47ed-8f52-3d4a84f4af6b admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:45:17.099314 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bf350f-3d36-4a2d-b803-ed83f9a76963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:45:17.100211 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf350f-3d36-4a2d-b803-ed83f9a76963 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:17.100498 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf350f-3d36-4a2d-b803-ed83f9a76963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:17.100776 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bf350f-3d36-4a2d-b803-ed83f9a76963 admin admin] PATH_INFO: `/v3/projects/ca94c8f670004e8b8d70d61b5aeee7c8/users/6524c022e8ff47c69753868b15bc63d3/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:17.115393 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bf350f-3d36-4a2d-b803-ed83f9a76963 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca94c8f670004e8b8d70d61b5aeee7c8, user_id=6524c022e8ff47c69753868b15bc63d3, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:45:17.118038 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bf350f-3d36-4a2d-b803-ed83f9a76963 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:45:17.177218 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1387/2726] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:45:17 2022] PUT /identity/v3/projects/ca94c8f670004e8b8d70d61b5aeee7c8/users/6524c022e8ff47c69753868b15bc63d3/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:45:17.182249 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215bb42a-766c-42e8-96a2-ec5f6445d2e7 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:17.182541 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215bb42a-766c-42e8-96a2-ec5f6445d2e7 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:17.182820 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215bb42a-766c-42e8-96a2-ec5f6445d2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:17.200409 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-215bb42a-766c-42e8-96a2-ec5f6445d2e7 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:45:17.201596 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-215bb42a-766c-42e8-96a2-ec5f6445d2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:45:17.259504 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1340/2727] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:45:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:45:17.268842 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bf350f-3d36-4a2d-b803-ed83f9a76963 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:45:17.270580 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-75bf350f-3d36-4a2d-b803-ed83f9a76963 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:45:17.325692 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bf350f-3d36-4a2d-b803-ed83f9a76963 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:45:17.329305 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a80f89a-2684-4c42-a540-51f98f3cfdb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:45:17.329945 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a80f89a-2684-4c42-a540-51f98f3cfdb4 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:17.330231 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a80f89a-2684-4c42-a540-51f98f3cfdb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:17.330511 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a80f89a-2684-4c42-a540-51f98f3cfdb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:17.339455 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a80f89a-2684-4c42-a540-51f98f3cfdb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:45:17.340902 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a80f89a-2684-4c42-a540-51f98f3cfdb4 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:45:17.350961 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1388/2728] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:45:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 17:45:18.568476 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc56aca-f740-4d2c-aa67-f62d5a4ea79c None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:18.568994 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc56aca-f740-4d2c-aa67-f62d5a4ea79c None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:18.569121 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc56aca-f740-4d2c-aa67-f62d5a4ea79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:18.596353 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-edc56aca-f740-4d2c-aa67-f62d5a4ea79c None None] MFA Rules not processed for user `6524c022e8ff47c69753868b15bc63d3`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:45:18.597598 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-edc56aca-f740-4d2c-aa67-f62d5a4ea79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:45:18.626069 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:45:18.665698 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1341/2729] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:45:18 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:45:18.670265 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a80f89a-2684-4c42-a540-51f98f3cfdb4 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:45:18.679990 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a80f89a-2684-4c42-a540-51f98f3cfdb4 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:45:18.684451 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734bd384-dcb9-4700-8f9d-791dcfe6cd1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:45:18.685090 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734bd384-dcb9-4700-8f9d-791dcfe6cd1e admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:18.685372 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734bd384-dcb9-4700-8f9d-791dcfe6cd1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:18.685646 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734bd384-dcb9-4700-8f9d-791dcfe6cd1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:18.686326 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734bd384-dcb9-4700-8f9d-791dcfe6cd1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:45:18.688271 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734bd384-dcb9-4700-8f9d-791dcfe6cd1e admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:45:18.783962 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1389/2730] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:45:18 2022] POST /identity/v3/projects => generated 373 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:45:18.788408 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc56aca-f740-4d2c-aa67-f62d5a4ea79c None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:45:18.799152 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc56aca-f740-4d2c-aa67-f62d5a4ea79c None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:45:18.854735 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ff9f18-175d-40e1-ada7-dc1abc31b86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:45:18.855448 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff9f18-175d-40e1-ada7-dc1abc31b86c admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:18.855757 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff9f18-175d-40e1-ada7-dc1abc31b86c admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:18.856035 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff9f18-175d-40e1-ada7-dc1abc31b86c admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:18.856720 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ff9f18-175d-40e1-ada7-dc1abc31b86c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:45:18.858630 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ff9f18-175d-40e1-ada7-dc1abc31b86c admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:45:18.983062 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1342/2731] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:45:18 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:45:18.986839 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734bd384-dcb9-4700-8f9d-791dcfe6cd1e admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:45:18.997347 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734bd384-dcb9-4700-8f9d-791dcfe6cd1e admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:45:19.011409 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad60ba-6868-4cad-b125-1d65043f8f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:45:19.011409 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad60ba-6868-4cad-b125-1d65043f8f65 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:19.011409 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad60ba-6868-4cad-b125-1d65043f8f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:19.011409 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fad60ba-6868-4cad-b125-1d65043f8f65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:19.011892 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fad60ba-6868-4cad-b125-1d65043f8f65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:45:19.011892 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fad60ba-6868-4cad-b125-1d65043f8f65 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:45:19.011892 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:45:19.017295 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1390/2732] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:45:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:45:19.021215 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ff9f18-175d-40e1-ada7-dc1abc31b86c admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:45:19.030963 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ff9f18-175d-40e1-ada7-dc1abc31b86c admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:45:19.035466 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4217fe7-25d6-415f-941a-bc6bfdb62166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:45:19.042287 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4217fe7-25d6-415f-941a-bc6bfdb62166 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:19.042287 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4217fe7-25d6-415f-941a-bc6bfdb62166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:19.042287 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4217fe7-25d6-415f-941a-bc6bfdb62166 admin admin] PATH_INFO: `/v3/projects/9e8eb07045034c35bfc091cd46d0c152/users/ababfd0f0c0a440192b3727c005ef6b6/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:19.053604 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4217fe7-25d6-415f-941a-bc6bfdb62166 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e8eb07045034c35bfc091cd46d0c152, user_id=ababfd0f0c0a440192b3727c005ef6b6, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:45:19.056334 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4217fe7-25d6-415f-941a-bc6bfdb62166 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:45:19.159349 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1343/2733] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:45:19 2022] PUT /identity/v3/projects/9e8eb07045034c35bfc091cd46d0c152/users/ababfd0f0c0a440192b3727c005ef6b6/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:45:19.163720 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad60ba-6868-4cad-b125-1d65043f8f65 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:45:19.173277 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fad60ba-6868-4cad-b125-1d65043f8f65 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:45:19.227619 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1f20f8-9d1b-484b-b7e9-5517a36a665c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:45:19.228307 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f20f8-9d1b-484b-b7e9-5517a36a665c admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:19.228590 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f20f8-9d1b-484b-b7e9-5517a36a665c admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:19.228864 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1f20f8-9d1b-484b-b7e9-5517a36a665c admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:19.229574 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1f20f8-9d1b-484b-b7e9-5517a36a665c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:45:19.231143 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1f20f8-9d1b-484b-b7e9-5517a36a665c admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:45:19.241854 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1391/2734] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:45:19 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:45:19.245158 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4217fe7-25d6-415f-941a-bc6bfdb62166 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:45:19.256063 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4217fe7-25d6-415f-941a-bc6bfdb62166 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:45:19.260663 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee90874-e151-4605-9d66-9b9a5bb4660a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:45:19.261568 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee90874-e151-4605-9d66-9b9a5bb4660a admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:19.261854 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee90874-e151-4605-9d66-9b9a5bb4660a admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:19.262130 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee90874-e151-4605-9d66-9b9a5bb4660a admin admin] PATH_INFO: `/v3/projects/9e8eb07045034c35bfc091cd46d0c152/users/ababfd0f0c0a440192b3727c005ef6b6/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:19.264117 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee90874-e151-4605-9d66-9b9a5bb4660a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e8eb07045034c35bfc091cd46d0c152, user_id=ababfd0f0c0a440192b3727c005ef6b6, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:45:19.266715 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee90874-e151-4605-9d66-9b9a5bb4660a admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:45:19.276544 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1344/2735] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:45:19 2022] PUT /identity/v3/projects/9e8eb07045034c35bfc091cd46d0c152/users/ababfd0f0c0a440192b3727c005ef6b6/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:45:20.476778 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e31e09-b97d-48da-a2d6-d316229cd6c0 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:20.477369 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e31e09-b97d-48da-a2d6-d316229cd6c0 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:20.477369 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e31e09-b97d-48da-a2d6-d316229cd6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:20.506383 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-c9e31e09-b97d-48da-a2d6-d316229cd6c0 None None] MFA Rules not processed for user `ababfd0f0c0a440192b3727c005ef6b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:45:20.507755 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c9e31e09-b97d-48da-a2d6-d316229cd6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:45:20.578406 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1392/2736] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:45:20 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:45:20.603529 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee90874-e151-4605-9d66-9b9a5bb4660a admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:45:20.605843 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-8ee90874-e151-4605-9d66-9b9a5bb4660a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:45:20.662381 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee90874-e151-4605-9d66-9b9a5bb4660a admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:45:20.665933 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c2ca7-9248-4b24-b643-711097ad048c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:45:20.666575 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c2ca7-9248-4b24-b643-711097ad048c service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:45:20.666859 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c2ca7-9248-4b24-b643-711097ad048c service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:45:20.667136 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3c2ca7-9248-4b24-b643-711097ad048c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:45:20.700380 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:45:20.723541 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c2ca7-9248-4b24-b643-711097ad048c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:45:20.724952 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3c2ca7-9248-4b24-b643-711097ad048c service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:45:20.733936 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1345/2737] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:45:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:45:30.575308 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=65 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:45:30.823680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=11 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:46:02.465420 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e31e09-b97d-48da-a2d6-d316229cd6c0 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:46:02.475213 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e31e09-b97d-48da-a2d6-d316229cd6c0 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:46:02.478846 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd0cdd-c44f-4c3a-870b-fb25a45295f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:46:02.479492 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd0cdd-c44f-4c3a-870b-fb25a45295f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:46:02.479808 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd0cdd-c44f-4c3a-870b-fb25a45295f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:46:02.480090 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44fd0cdd-c44f-4c3a-870b-fb25a45295f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:46:02.481899 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-44fd0cdd-c44f-4c3a-870b-fb25a45295f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:46:02.548624 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fd0cdd-c44f-4c3a-870b-fb25a45295f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:46:02.549980 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44fd0cdd-c44f-4c3a-870b-fb25a45295f7 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:46:02.567494 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1393/2738] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:46:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:46:12.564978 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=36 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:47:47.126255 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c2ca7-9248-4b24-b643-711097ad048c service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:47:47.127667 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-fc3c2ca7-9248-4b24-b643-711097ad048c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:47:47.253987 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3c2ca7-9248-4b24-b643-711097ad048c service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:47:47.258935 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5ea35f-9c41-4aed-b39a-9f22e745884f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76ca877bee9041d28ad41e45ef3dd18c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:47:47.260142 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ea35f-9c41-4aed-b39a-9f22e745884f service placement] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:47:47.260423 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ea35f-9c41-4aed-b39a-9f22e745884f service placement] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:47:47.260693 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ea35f-9c41-4aed-b39a-9f22e745884f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:47:47.262451 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-2f5ea35f-9c41-4aed-b39a-9f22e745884f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:47:47.273440 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5ea35f-9c41-4aed-b39a-9f22e745884f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:47:47.275612 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5ea35f-9c41-4aed-b39a-9f22e745884f service placement] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:47:47.285357 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1346/2739] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:47:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:47:57.281137 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=58 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:49:51.628642 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd0cdd-c44f-4c3a-870b-fb25a45295f7 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:49:51.648120 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44fd0cdd-c44f-4c3a-870b-fb25a45295f7 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:49:51.651805 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67107ddb-f3ab-4cd6-9e27-1c91f88d30f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:49:51.652523 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67107ddb-f3ab-4cd6-9e27-1c91f88d30f0 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:49:51.652806 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67107ddb-f3ab-4cd6-9e27-1c91f88d30f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:49:51.653080 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67107ddb-f3ab-4cd6-9e27-1c91f88d30f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:49:51.654910 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-67107ddb-f3ab-4cd6-9e27-1c91f88d30f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:49:51.713810 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67107ddb-f3ab-4cd6-9e27-1c91f88d30f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:49:51.715248 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67107ddb-f3ab-4cd6-9e27-1c91f88d30f0 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:49:51.724294 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1394/2740] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:49:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:50:01.720134 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=33 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:50:20.185677 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5ea35f-9c41-4aed-b39a-9f22e745884f service placement] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:50:20.205571 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5ea35f-9c41-4aed-b39a-9f22e745884f service placement] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:50:20.209103 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec80c2-b0a6-4748-b112-920c46f4238c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:50:20.209738 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec80c2-b0a6-4748-b112-920c46f4238c service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:50:20.210019 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec80c2-b0a6-4748-b112-920c46f4238c service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:50:20.210292 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ec80c2-b0a6-4748-b112-920c46f4238c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:50:20.218450 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ec80c2-b0a6-4748-b112-920c46f4238c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:50:20.219901 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ec80c2-b0a6-4748-b112-920c46f4238c service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:50:20.228529 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1347/2741] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:50:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:50:30.224611 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=11 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:51:02.317910 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67107ddb-f3ab-4cd6-9e27-1c91f88d30f0 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:02.327037 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67107ddb-f3ab-4cd6-9e27-1c91f88d30f0 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:02.330542 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de697a-a410-4442-93a9-23f1e1dd3c2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:02.331172 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de697a-a410-4442-93a9-23f1e1dd3c2f service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:02.331453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de697a-a410-4442-93a9-23f1e1dd3c2f service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:02.331744 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81de697a-a410-4442-93a9-23f1e1dd3c2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:02.339815 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81de697a-a410-4442-93a9-23f1e1dd3c2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:02.341216 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81de697a-a410-4442-93a9-23f1e1dd3c2f service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:02.349838 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1395/2742] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:51:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:51:05.533311 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec80c2-b0a6-4748-b112-920c46f4238c service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:05.534723 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-67ec80c2-b0a6-4748-b112-920c46f4238c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:05.593445 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ec80c2-b0a6-4748-b112-920c46f4238c service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:05.596956 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538212ff-ca82-4165-9826-0c8d1060cc42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:05.597593 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538212ff-ca82-4165-9826-0c8d1060cc42 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:05.597875 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538212ff-ca82-4165-9826-0c8d1060cc42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:05.598150 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538212ff-ca82-4165-9826-0c8d1060cc42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:05.600480 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-538212ff-ca82-4165-9826-0c8d1060cc42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:05.660519 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538212ff-ca82-4165-9826-0c8d1060cc42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:05.662567 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538212ff-ca82-4165-9826-0c8d1060cc42 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:05.672449 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1348/2743] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:51:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:51:09.098631 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de697a-a410-4442-93a9-23f1e1dd3c2f service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:09.110503 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81de697a-a410-4442-93a9-23f1e1dd3c2f service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:09.116063 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8ab1aa-fb50-47e2-b8ba-27a6376bfd07 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76ca877bee9041d28ad41e45ef3dd18c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:09.116715 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ab1aa-fb50-47e2-b8ba-27a6376bfd07 service placement] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:09.117004 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ab1aa-fb50-47e2-b8ba-27a6376bfd07 service placement] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:09.117287 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ab1aa-fb50-47e2-b8ba-27a6376bfd07 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:09.119134 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-ef8ab1aa-fb50-47e2-b8ba-27a6376bfd07 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:09.131718 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8ab1aa-fb50-47e2-b8ba-27a6376bfd07 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:09.133321 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8ab1aa-fb50-47e2-b8ba-27a6376bfd07 service placement] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:09.143930 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1396/2744] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:51:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:51:09.247072 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538212ff-ca82-4165-9826-0c8d1060cc42 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:09.249091 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-538212ff-ca82-4165-9826-0c8d1060cc42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:09.263026 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538212ff-ca82-4165-9826-0c8d1060cc42 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:09.267696 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04ac362-83ac-4e62-b89f-84afbdac1e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:09.268595 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ac362-83ac-4e62-b89f-84afbdac1e0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:09.268878 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ac362-83ac-4e62-b89f-84afbdac1e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:09.269153 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04ac362-83ac-4e62-b89f-84afbdac1e0f admin admin] PATH_INFO: `/v3/users/ae705e3500914acda9d7a123b4c36298` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:09.270331 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04ac362-83ac-4e62-b89f-84afbdac1e0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae705e3500914acda9d7a123b4c36298)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:09.272272 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04ac362-83ac-4e62-b89f-84afbdac1e0f admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:09.488339 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-f04ac362-83ac-4e62-b89f-84afbdac1e0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae705e3500914acda9d7a123b4c36298'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:09.501173 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-f04ac362-83ac-4e62-b89f-84afbdac1e0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae705e3500914acda9d7a123b4c36298'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:09.508040 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-f04ac362-83ac-4e62-b89f-84afbdac1e0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae705e3500914acda9d7a123b4c36298'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:09.508735 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-f04ac362-83ac-4e62-b89f-84afbdac1e0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae705e3500914acda9d7a123b4c36298'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:09.600877 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-f04ac362-83ac-4e62-b89f-84afbdac1e0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae705e3500914acda9d7a123b4c36298'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:09.607437 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1349/2745] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:51:09 2022] DELETE /identity/v3/users/ae705e3500914acda9d7a123b4c36298 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:51:09.865567 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8ab1aa-fb50-47e2-b8ba-27a6376bfd07 service placement] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:09.869671 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8ab1aa-fb50-47e2-b8ba-27a6376bfd07 service placement] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:09.926930 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda7465-8a84-4b6a-b5c3-60dd1579c099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:09.927827 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda7465-8a84-4b6a-b5c3-60dd1579c099 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:09.928112 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda7465-8a84-4b6a-b5c3-60dd1579c099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:09.928411 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda7465-8a84-4b6a-b5c3-60dd1579c099 admin admin] PATH_INFO: `/v3/projects/cb01cd40add5405490508d2d14d1902d` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:09.938740 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda7465-8a84-4b6a-b5c3-60dd1579c099 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb01cd40add5405490508d2d14d1902d)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:09.940840 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda7465-8a84-4b6a-b5c3-60dd1579c099 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:10.034298 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1eda7465-8a84-4b6a-b5c3-60dd1579c099 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb01cd40add5405490508d2d14d1902d', 'request_id': 'req-1eda7465-8a84-4b6a-b5c3-60dd1579c099'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:10.041499 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1eda7465-8a84-4b6a-b5c3-60dd1579c099 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb01cd40add5405490508d2d14d1902d', 'request_id': 'req-1eda7465-8a84-4b6a-b5c3-60dd1579c099'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:10.046909 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1eda7465-8a84-4b6a-b5c3-60dd1579c099 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb01cd40add5405490508d2d14d1902d', 'request_id': 'req-1eda7465-8a84-4b6a-b5c3-60dd1579c099'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:10.107788 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1eda7465-8a84-4b6a-b5c3-60dd1579c099 admin admin] The token cache is being invalidate because project cb01cd40add5405490508d2d14d1902d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:51:10.108465 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1eda7465-8a84-4b6a-b5c3-60dd1579c099 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:10.108762 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-1eda7465-8a84-4b6a-b5c3-60dd1579c099 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:10.110307 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1397/2746] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:51:09 2022] DELETE /identity/v3/projects/cb01cd40add5405490508d2d14d1902d => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:51:10.116626 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850df84e-6c89-4a2e-a95d-566e951c6358 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:10.116929 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850df84e-6c89-4a2e-a95d-566e951c6358 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:10.117240 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850df84e-6c89-4a2e-a95d-566e951c6358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:10.135968 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-850df84e-6c89-4a2e-a95d-566e951c6358 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:51:10.137220 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-850df84e-6c89-4a2e-a95d-566e951c6358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:10.156072 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:51:10.197765 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1350/2747] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:51:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:51:10.206070 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c441a8-af53-4b78-aaaa-13124001fe00 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:10.206367 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c441a8-af53-4b78-aaaa-13124001fe00 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:10.206655 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c441a8-af53-4b78-aaaa-13124001fe00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:10.225187 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-24c441a8-af53-4b78-aaaa-13124001fe00 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:51:10.226392 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-24c441a8-af53-4b78-aaaa-13124001fe00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:10.234401 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1398/2748] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:51:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:51:10.238487 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850df84e-6c89-4a2e-a95d-566e951c6358 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:10.248250 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850df84e-6c89-4a2e-a95d-566e951c6358 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:10.252881 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac1f0ab-d8dd-445a-a886-26cc14b1a33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:10.253543 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac1f0ab-d8dd-445a-a886-26cc14b1a33d admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:10.253835 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac1f0ab-d8dd-445a-a886-26cc14b1a33d admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:10.254121 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac1f0ab-d8dd-445a-a886-26cc14b1a33d admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:10.254855 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac1f0ab-d8dd-445a-a886-26cc14b1a33d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:51:10.255169 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac1f0ab-d8dd-445a-a886-26cc14b1a33d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:10.257367 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac1f0ab-d8dd-445a-a886-26cc14b1a33d admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:10.268790 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1351/2749] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:51:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 17:51:10.273508 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c441a8-af53-4b78-aaaa-13124001fe00 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:10.282503 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c441a8-af53-4b78-aaaa-13124001fe00 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:10.287117 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d47185-5b4b-4895-b049-50ec57d01712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:10.287701 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d47185-5b4b-4895-b049-50ec57d01712 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:10.288011 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d47185-5b4b-4895-b049-50ec57d01712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:10.288293 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d47185-5b4b-4895-b049-50ec57d01712 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:10.288978 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d47185-5b4b-4895-b049-50ec57d01712 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:10.291136 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d47185-5b4b-4895-b049-50ec57d01712 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:10.378061 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1399/2750] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:51:10 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:51:10.382716 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac1f0ab-d8dd-445a-a886-26cc14b1a33d admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:10.392879 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac1f0ab-d8dd-445a-a886-26cc14b1a33d admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:10.450472 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a6657-1e28-48ad-9907-1ed1a1d32e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:10.451133 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a6657-1e28-48ad-9907-1ed1a1d32e48 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:10.451420 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a6657-1e28-48ad-9907-1ed1a1d32e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:10.451694 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a6657-1e28-48ad-9907-1ed1a1d32e48 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:10.452407 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901a6657-1e28-48ad-9907-1ed1a1d32e48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:10.454317 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901a6657-1e28-48ad-9907-1ed1a1d32e48 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:10.560784 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1352/2751] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:51:10 2022] POST /identity/v3/users => generated 287 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:51:10.564944 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d47185-5b4b-4895-b049-50ec57d01712 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:10.576076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d47185-5b4b-4895-b049-50ec57d01712 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:10.580678 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3deaf8-0525-4509-bddd-067f08ad22af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:10.581399 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3deaf8-0525-4509-bddd-067f08ad22af admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:10.581687 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3deaf8-0525-4509-bddd-067f08ad22af admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:10.581967 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3deaf8-0525-4509-bddd-067f08ad22af admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:10.582698 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3deaf8-0525-4509-bddd-067f08ad22af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:10.584323 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3deaf8-0525-4509-bddd-067f08ad22af admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:10.594600 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1400/2752] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:51:10.598449 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a6657-1e28-48ad-9907-1ed1a1d32e48 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:10.608366 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a6657-1e28-48ad-9907-1ed1a1d32e48 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:10.612922 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646e1498-f3d8-4b79-9154-eff276006b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:10.613576 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e1498-f3d8-4b79-9154-eff276006b1e admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:10.613861 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e1498-f3d8-4b79-9154-eff276006b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:10.614141 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646e1498-f3d8-4b79-9154-eff276006b1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:10.615375 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646e1498-f3d8-4b79-9154-eff276006b1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:10.617017 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646e1498-f3d8-4b79-9154-eff276006b1e admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:10.626606 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1353/2753] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:51:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:51:10.631148 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3deaf8-0525-4509-bddd-067f08ad22af admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:10.640851 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3deaf8-0525-4509-bddd-067f08ad22af admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:10.645580 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3f5eac-badb-4550-8958-ba2c86ecc90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:10.646445 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f5eac-badb-4550-8958-ba2c86ecc90d admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:10.646728 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f5eac-badb-4550-8958-ba2c86ecc90d admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:10.647000 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3f5eac-badb-4550-8958-ba2c86ecc90d admin admin] PATH_INFO: `/v3/projects/c68af44aa59f44deb69866631720b64b/users/6273c7f9d34e46f6b6581441c0384cc7/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:10.658497 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:51:10.661372 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3f5eac-badb-4550-8958-ba2c86ecc90d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c68af44aa59f44deb69866631720b64b, user_id=6273c7f9d34e46f6b6581441c0384cc7, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:10.663991 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3f5eac-badb-4550-8958-ba2c86ecc90d admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:10.766368 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1401/2754] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:51:10 2022] PUT /identity/v3/projects/c68af44aa59f44deb69866631720b64b/users/6273c7f9d34e46f6b6581441c0384cc7/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:51:10.783682 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef463e1-6088-49ed-a139-9c882f294a86 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:10.784183 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef463e1-6088-49ed-a139-9c882f294a86 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:10.784477 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef463e1-6088-49ed-a139-9c882f294a86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:10.813874 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-cef463e1-6088-49ed-a139-9c882f294a86 None None] MFA Rules not processed for user `6273c7f9d34e46f6b6581441c0384cc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:51:10.815059 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-cef463e1-6088-49ed-a139-9c882f294a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:10.878636 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1354/2755] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:51:10 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:51:10.891483 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3f5eac-badb-4550-8958-ba2c86ecc90d admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:10.893190 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-2d3f5eac-badb-4550-8958-ba2c86ecc90d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:10.998645 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3f5eac-badb-4550-8958-ba2c86ecc90d admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:11.004394 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfda4cc-89c0-426b-9f50-fe44ad4e0049 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:11.005158 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfda4cc-89c0-426b-9f50-fe44ad4e0049 service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:11.005550 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfda4cc-89c0-426b-9f50-fe44ad4e0049 service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:11.005896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfda4cc-89c0-426b-9f50-fe44ad4e0049 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:11.014233 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfda4cc-89c0-426b-9f50-fe44ad4e0049 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:11.015926 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfda4cc-89c0-426b-9f50-fe44ad4e0049 service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:11.024957 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1402/2756] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:51:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:51:11.116280 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef463e1-6088-49ed-a139-9c882f294a86 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:11.118199 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-cef463e1-6088-49ed-a139-9c882f294a86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:11.143947 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:51:11.191756 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef463e1-6088-49ed-a139-9c882f294a86 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:11.196297 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2262330-1fd4-41f4-a374-c5b788778321 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:11.196951 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2262330-1fd4-41f4-a374-c5b788778321 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:11.197234 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2262330-1fd4-41f4-a374-c5b788778321 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:11.197565 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2262330-1fd4-41f4-a374-c5b788778321 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:11.199508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e2262330-1fd4-41f4-a374-c5b788778321 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:11.213431 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2262330-1fd4-41f4-a374-c5b788778321 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:11.214868 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2262330-1fd4-41f4-a374-c5b788778321 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:11.226002 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1355/2757] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:51:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:51:11.292864 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfda4cc-89c0-426b-9f50-fe44ad4e0049 service nova] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:11.302913 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfda4cc-89c0-426b-9f50-fe44ad4e0049 service nova] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:11.306441 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7128118-9394-4f11-9399-d18685d1588b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:11.307072 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7128118-9394-4f11-9399-d18685d1588b service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:11.307353 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7128118-9394-4f11-9399-d18685d1588b service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:11.307625 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7128118-9394-4f11-9399-d18685d1588b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:11.309469 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-c7128118-9394-4f11-9399-d18685d1588b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:11.323035 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7128118-9394-4f11-9399-d18685d1588b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:11.324419 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7128118-9394-4f11-9399-d18685d1588b service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:11.335054 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1403/2758] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:51:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:51:11.409859 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2262330-1fd4-41f4-a374-c5b788778321 service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:11.472269 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2262330-1fd4-41f4-a374-c5b788778321 service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:11.476947 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d382206-2c9c-4c0a-b3a0-b4629039c038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:11.477818 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d382206-2c9c-4c0a-b3a0-b4629039c038 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:11.478101 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d382206-2c9c-4c0a-b3a0-b4629039c038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:11.478376 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d382206-2c9c-4c0a-b3a0-b4629039c038 admin admin] PATH_INFO: `/v3/users/6273c7f9d34e46f6b6581441c0384cc7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:11.479683 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d382206-2c9c-4c0a-b3a0-b4629039c038 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6273c7f9d34e46f6b6581441c0384cc7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:11.481709 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d382206-2c9c-4c0a-b3a0-b4629039c038 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:11.653951 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-4d382206-2c9c-4c0a-b3a0-b4629039c038 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6273c7f9d34e46f6b6581441c0384cc7'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:11.666508 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-4d382206-2c9c-4c0a-b3a0-b4629039c038 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6273c7f9d34e46f6b6581441c0384cc7'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:11.673321 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-4d382206-2c9c-4c0a-b3a0-b4629039c038 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6273c7f9d34e46f6b6581441c0384cc7'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:11.673998 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-4d382206-2c9c-4c0a-b3a0-b4629039c038 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6273c7f9d34e46f6b6581441c0384cc7'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:11.720153 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-4d382206-2c9c-4c0a-b3a0-b4629039c038 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6273c7f9d34e46f6b6581441c0384cc7'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:11.726714 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1356/2759] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:51:11 2022] DELETE /identity/v3/users/6273c7f9d34e46f6b6581441c0384cc7 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:51:11.731275 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a69426-0308-49a4-bb9b-2165fbad3078 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:11.731573 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a69426-0308-49a4-bb9b-2165fbad3078 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:11.731881 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a69426-0308-49a4-bb9b-2165fbad3078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:11.750964 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-89a69426-0308-49a4-bb9b-2165fbad3078 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:51:11.752168 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-89a69426-0308-49a4-bb9b-2165fbad3078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:11.812175 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1404/2760] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:51:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:51:11.821614 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d382206-2c9c-4c0a-b3a0-b4629039c038 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:11.823024 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4d382206-2c9c-4c0a-b3a0-b4629039c038 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:11.857896 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:51:11.890235 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d382206-2c9c-4c0a-b3a0-b4629039c038 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:11.894931 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59178b70-e086-4624-92c0-9ef6c6bb07d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:11.896885 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59178b70-e086-4624-92c0-9ef6c6bb07d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:11.896885 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59178b70-e086-4624-92c0-9ef6c6bb07d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:11.896885 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59178b70-e086-4624-92c0-9ef6c6bb07d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:11.906600 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59178b70-e086-4624-92c0-9ef6c6bb07d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:11.908050 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59178b70-e086-4624-92c0-9ef6c6bb07d5 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:11.919027 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1357/2761] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:51:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:51:12.310099 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a69426-0308-49a4-bb9b-2165fbad3078 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:12.312247 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-89a69426-0308-49a4-bb9b-2165fbad3078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:12.324948 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a69426-0308-49a4-bb9b-2165fbad3078 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:12.329518 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f877a-db51-4853-b081-36a2d940966e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:12.330374 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f877a-db51-4853-b081-36a2d940966e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:12.330668 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f877a-db51-4853-b081-36a2d940966e admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:12.330969 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f877a-db51-4853-b081-36a2d940966e admin admin] PATH_INFO: `/v3/projects/c68af44aa59f44deb69866631720b64b` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:12.332589 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f877a-db51-4853-b081-36a2d940966e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c68af44aa59f44deb69866631720b64b)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:12.335104 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f877a-db51-4853-b081-36a2d940966e admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:12.469008 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:51:12.480405 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-354f877a-db51-4853-b081-36a2d940966e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c68af44aa59f44deb69866631720b64b', 'request_id': 'req-354f877a-db51-4853-b081-36a2d940966e'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:12.486627 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-354f877a-db51-4853-b081-36a2d940966e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c68af44aa59f44deb69866631720b64b', 'request_id': 'req-354f877a-db51-4853-b081-36a2d940966e'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:12.491985 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-354f877a-db51-4853-b081-36a2d940966e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c68af44aa59f44deb69866631720b64b', 'request_id': 'req-354f877a-db51-4853-b081-36a2d940966e'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:12.571324 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-354f877a-db51-4853-b081-36a2d940966e admin admin] The token cache is being invalidate because project c68af44aa59f44deb69866631720b64b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:51:12.571811 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-354f877a-db51-4853-b081-36a2d940966e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:12.572884 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-354f877a-db51-4853-b081-36a2d940966e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:12.574453 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1405/2762] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:51:12 2022] DELETE /identity/v3/projects/c68af44aa59f44deb69866631720b64b => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:51:12.579289 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf7f277-dd05-40ea-a1a2-0d49ddf5d7f5 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:12.586332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf7f277-dd05-40ea-a1a2-0d49ddf5d7f5 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:12.586332 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf7f277-dd05-40ea-a1a2-0d49ddf5d7f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:12.598265 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-6cf7f277-dd05-40ea-a1a2-0d49ddf5d7f5 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:51:12.599460 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-6cf7f277-dd05-40ea-a1a2-0d49ddf5d7f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:12.657795 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1358/2763] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:51:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:51:12.665544 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d5114-b436-476f-9dcb-550980e2f014 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:12.665831 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d5114-b436-476f-9dcb-550980e2f014 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:12.666105 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376d5114-b436-476f-9dcb-550980e2f014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:12.684258 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-376d5114-b436-476f-9dcb-550980e2f014 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:51:12.685437 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-376d5114-b436-476f-9dcb-550980e2f014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:12.693844 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1406/2764] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:51:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:51:12.697875 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf7f277-dd05-40ea-a1a2-0d49ddf5d7f5 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:12.707675 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf7f277-dd05-40ea-a1a2-0d49ddf5d7f5 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:12.712394 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787893db-8704-4920-9fc8-c33c733e8c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:12.713050 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787893db-8704-4920-9fc8-c33c733e8c7d admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:12.713339 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787893db-8704-4920-9fc8-c33c733e8c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:12.713622 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787893db-8704-4920-9fc8-c33c733e8c7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:12.714356 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787893db-8704-4920-9fc8-c33c733e8c7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:51:12.714666 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787893db-8704-4920-9fc8-c33c733e8c7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:12.716338 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787893db-8704-4920-9fc8-c33c733e8c7d admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:12.728220 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1359/2765] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:51:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:51:12.732529 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376d5114-b436-476f-9dcb-550980e2f014 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:12.742156 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376d5114-b436-476f-9dcb-550980e2f014 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:12.746709 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e899b34-b646-4fa0-87d6-d5b88cc691b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:12.747464 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e899b34-b646-4fa0-87d6-d5b88cc691b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:12.747808 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e899b34-b646-4fa0-87d6-d5b88cc691b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:12.748084 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e899b34-b646-4fa0-87d6-d5b88cc691b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:12.748756 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e899b34-b646-4fa0-87d6-d5b88cc691b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:12.750626 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e899b34-b646-4fa0-87d6-d5b88cc691b6 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:12.876081 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1407/2766] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:51:12 2022] POST /identity/v3/projects => generated 367 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:51:12.881050 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787893db-8704-4920-9fc8-c33c733e8c7d admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:12.890982 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787893db-8704-4920-9fc8-c33c733e8c7d admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:12.946785 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91746b8e-7d23-4a6f-b83c-806360cca7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:12.947451 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91746b8e-7d23-4a6f-b83c-806360cca7f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:12.947753 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91746b8e-7d23-4a6f-b83c-806360cca7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:12.948047 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91746b8e-7d23-4a6f-b83c-806360cca7f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:12.948740 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91746b8e-7d23-4a6f-b83c-806360cca7f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:12.950661 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91746b8e-7d23-4a6f-b83c-806360cca7f8 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:13.046662 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1360/2767] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:51:12 2022] POST /identity/v3/users => generated 284 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:51:13.050539 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e899b34-b646-4fa0-87d6-d5b88cc691b6 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:13.061415 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e899b34-b646-4fa0-87d6-d5b88cc691b6 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:13.066008 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534a09bf-7e07-407a-8c04-837791080293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:13.066660 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a09bf-7e07-407a-8c04-837791080293 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:13.066947 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a09bf-7e07-407a-8c04-837791080293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:13.067222 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534a09bf-7e07-407a-8c04-837791080293 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:13.068014 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534a09bf-7e07-407a-8c04-837791080293 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:13.069596 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534a09bf-7e07-407a-8c04-837791080293 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:13.079695 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1408/2768] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:51:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:51:13.083478 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91746b8e-7d23-4a6f-b83c-806360cca7f8 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:13.093286 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91746b8e-7d23-4a6f-b83c-806360cca7f8 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:13.097889 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6457534b-ee82-4cda-90b3-5b84187060d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:13.098536 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457534b-ee82-4cda-90b3-5b84187060d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:13.098824 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457534b-ee82-4cda-90b3-5b84187060d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:13.099616 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6457534b-ee82-4cda-90b3-5b84187060d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:13.100383 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6457534b-ee82-4cda-90b3-5b84187060d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:13.101982 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6457534b-ee82-4cda-90b3-5b84187060d8 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:13.111912 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1361/2769] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:51:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:51:13.116458 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534a09bf-7e07-407a-8c04-837791080293 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:13.126399 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534a09bf-7e07-407a-8c04-837791080293 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:13.130963 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ffe04-6676-48f1-aabd-5fc3ddc5c357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:13.131862 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ffe04-6676-48f1-aabd-5fc3ddc5c357 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:13.132145 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ffe04-6676-48f1-aabd-5fc3ddc5c357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:13.132419 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509ffe04-6676-48f1-aabd-5fc3ddc5c357 admin admin] PATH_INFO: `/v3/projects/3b9fcd5b09b9443f9a00e3a7dc85a10a/users/975a1e6ec2ed46c693cfe82635fa5004/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:13.146335 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509ffe04-6676-48f1-aabd-5fc3ddc5c357 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b9fcd5b09b9443f9a00e3a7dc85a10a, user_id=975a1e6ec2ed46c693cfe82635fa5004, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:13.149726 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509ffe04-6676-48f1-aabd-5fc3ddc5c357 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:13.216905 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1409/2770] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:51:13 2022] PUT /identity/v3/projects/3b9fcd5b09b9443f9a00e3a7dc85a10a/users/975a1e6ec2ed46c693cfe82635fa5004/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:51:13.228902 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0120b5-5e4e-49ee-9241-f55a39045cf1 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:13.230095 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0120b5-5e4e-49ee-9241-f55a39045cf1 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:13.230095 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0120b5-5e4e-49ee-9241-f55a39045cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:13.255743 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-4d0120b5-5e4e-49ee-9241-f55a39045cf1 None None] MFA Rules not processed for user `975a1e6ec2ed46c693cfe82635fa5004`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:51:13.257004 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4d0120b5-5e4e-49ee-9241-f55a39045cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:13.275136 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:51:13.323043 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1362/2771] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:51:13 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:51:13.327382 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ffe04-6676-48f1-aabd-5fc3ddc5c357 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:13.387925 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509ffe04-6676-48f1-aabd-5fc3ddc5c357 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:13.392572 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9fdd87-f3b0-4b92-ab04-dd62999a7c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:13.393229 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9fdd87-f3b0-4b92-ab04-dd62999a7c8e admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:13.393515 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9fdd87-f3b0-4b92-ab04-dd62999a7c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:13.393794 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9fdd87-f3b0-4b92-ab04-dd62999a7c8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:13.394473 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9fdd87-f3b0-4b92-ab04-dd62999a7c8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:13.396904 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9fdd87-f3b0-4b92-ab04-dd62999a7c8e admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:13.458155 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1410/2772] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:51:13 2022] POST /identity/v3/projects => generated 367 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:51:13.462811 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0120b5-5e4e-49ee-9241-f55a39045cf1 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:13.473730 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0120b5-5e4e-49ee-9241-f55a39045cf1 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:13.533411 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba3d42-c3d4-4f75-a176-25f57b8ec630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:13.534087 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba3d42-c3d4-4f75-a176-25f57b8ec630 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:13.534393 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba3d42-c3d4-4f75-a176-25f57b8ec630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:13.534647 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba3d42-c3d4-4f75-a176-25f57b8ec630 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:13.535343 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba3d42-c3d4-4f75-a176-25f57b8ec630 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:13.537981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dba3d42-c3d4-4f75-a176-25f57b8ec630 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:13.589096 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1363/2773] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:51:13 2022] POST /identity/v3/users => generated 290 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:51:13.593074 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9fdd87-f3b0-4b92-ab04-dd62999a7c8e admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:13.604416 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9fdd87-f3b0-4b92-ab04-dd62999a7c8e admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:13.611225 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3eec35-e346-4618-b2e3-94aca122a05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:13.612384 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3eec35-e346-4618-b2e3-94aca122a05f admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:13.612896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3eec35-e346-4618-b2e3-94aca122a05f admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:13.613403 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3eec35-e346-4618-b2e3-94aca122a05f admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:13.614754 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3eec35-e346-4618-b2e3-94aca122a05f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:13.617107 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3eec35-e346-4618-b2e3-94aca122a05f admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:13.627801 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1411/2774] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:51:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:51:13.632030 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba3d42-c3d4-4f75-a176-25f57b8ec630 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:13.641932 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba3d42-c3d4-4f75-a176-25f57b8ec630 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:13.647046 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff8ccb-b6c7-4db6-a437-41fc0870118d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:13.647951 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff8ccb-b6c7-4db6-a437-41fc0870118d admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:13.648236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff8ccb-b6c7-4db6-a437-41fc0870118d admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:13.648510 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eff8ccb-b6c7-4db6-a437-41fc0870118d admin admin] PATH_INFO: `/v3/projects/5f3178904ad2418b9aea21cb31d8dd56/users/012e850179da48dbb13d2c2728172d88/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:13.663670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eff8ccb-b6c7-4db6-a437-41fc0870118d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f3178904ad2418b9aea21cb31d8dd56, user_id=012e850179da48dbb13d2c2728172d88, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:13.666636 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eff8ccb-b6c7-4db6-a437-41fc0870118d admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:13.761063 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1364/2775] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:51:13 2022] PUT /identity/v3/projects/5f3178904ad2418b9aea21cb31d8dd56/users/012e850179da48dbb13d2c2728172d88/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:51:13.765164 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3eec35-e346-4618-b2e3-94aca122a05f admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:13.775920 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3eec35-e346-4618-b2e3-94aca122a05f admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:13.805443 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:51:13.832542 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6956b4-0101-4b06-8f45-9eec8b926408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:13.833233 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6956b4-0101-4b06-8f45-9eec8b926408 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:13.834149 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6956b4-0101-4b06-8f45-9eec8b926408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:13.834426 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6956b4-0101-4b06-8f45-9eec8b926408 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:13.835170 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6956b4-0101-4b06-8f45-9eec8b926408 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:13.836783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6956b4-0101-4b06-8f45-9eec8b926408 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:13.849815 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1412/2776] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:51:13 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:51:13.851012 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff8ccb-b6c7-4db6-a437-41fc0870118d admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:13.861319 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eff8ccb-b6c7-4db6-a437-41fc0870118d admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:13.866007 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353188ff-97ef-40c9-8cef-746fad119687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:13.866887 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353188ff-97ef-40c9-8cef-746fad119687 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:13.867167 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353188ff-97ef-40c9-8cef-746fad119687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:13.867442 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353188ff-97ef-40c9-8cef-746fad119687 admin admin] PATH_INFO: `/v3/projects/5f3178904ad2418b9aea21cb31d8dd56/users/012e850179da48dbb13d2c2728172d88/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:13.869421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353188ff-97ef-40c9-8cef-746fad119687 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f3178904ad2418b9aea21cb31d8dd56, user_id=012e850179da48dbb13d2c2728172d88, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:13.872047 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353188ff-97ef-40c9-8cef-746fad119687 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:13.891413 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1365/2777] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:51:13 2022] PUT /identity/v3/projects/5f3178904ad2418b9aea21cb31d8dd56/users/012e850179da48dbb13d2c2728172d88/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:51:13.902243 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9006987-2066-4dc0-a445-6e435199b5c6 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:13.902531 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9006987-2066-4dc0-a445-6e435199b5c6 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:13.902810 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9006987-2066-4dc0-a445-6e435199b5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:13.928514 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-e9006987-2066-4dc0-a445-6e435199b5c6 None None] MFA Rules not processed for user `012e850179da48dbb13d2c2728172d88`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:51:13.929690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-e9006987-2066-4dc0-a445-6e435199b5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:13.997233 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1413/2778] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:51:13 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:51:14.008510 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353188ff-97ef-40c9-8cef-746fad119687 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:14.010175 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-353188ff-97ef-40c9-8cef-746fad119687 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:14.066427 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353188ff-97ef-40c9-8cef-746fad119687 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:14.070791 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48802f61-7b7a-4168-8e21-fc3f200d5b1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:14.071458 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48802f61-7b7a-4168-8e21-fc3f200d5b1c service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:14.071766 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48802f61-7b7a-4168-8e21-fc3f200d5b1c service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:14.072047 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48802f61-7b7a-4168-8e21-fc3f200d5b1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:14.082343 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48802f61-7b7a-4168-8e21-fc3f200d5b1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:14.084396 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48802f61-7b7a-4168-8e21-fc3f200d5b1c service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:14.093895 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1366/2779] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:51:14 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:51:16.691789 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9006987-2066-4dc0-a445-6e435199b5c6 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:16.755408 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9006987-2066-4dc0-a445-6e435199b5c6 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:16.760527 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869921fa-a529-46a8-9975-f954b7bd31a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:16.761392 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869921fa-a529-46a8-9975-f954b7bd31a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:16.761673 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869921fa-a529-46a8-9975-f954b7bd31a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:16.761950 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869921fa-a529-46a8-9975-f954b7bd31a8 admin admin] PATH_INFO: `/v3/users/975a1e6ec2ed46c693cfe82635fa5004` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:16.763152 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869921fa-a529-46a8-9975-f954b7bd31a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=975a1e6ec2ed46c693cfe82635fa5004)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:16.765081 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869921fa-a529-46a8-9975-f954b7bd31a8 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:16.991159 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-869921fa-a529-46a8-9975-f954b7bd31a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '975a1e6ec2ed46c693cfe82635fa5004'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:16.996424 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:51:16.998328 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-869921fa-a529-46a8-9975-f954b7bd31a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '975a1e6ec2ed46c693cfe82635fa5004'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:16.999002 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-869921fa-a529-46a8-9975-f954b7bd31a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '975a1e6ec2ed46c693cfe82635fa5004'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:17.010361 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-869921fa-a529-46a8-9975-f954b7bd31a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '975a1e6ec2ed46c693cfe82635fa5004'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:17.069399 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-869921fa-a529-46a8-9975-f954b7bd31a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '975a1e6ec2ed46c693cfe82635fa5004'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:17.076737 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1414/2780] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:51:16 2022] DELETE /identity/v3/users/975a1e6ec2ed46c693cfe82635fa5004 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:51:17.082967 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac46fbd-7d96-44bf-ac01-ee49ebae2ec1 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:17.083483 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac46fbd-7d96-44bf-ac01-ee49ebae2ec1 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:17.083603 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac46fbd-7d96-44bf-ac01-ee49ebae2ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:17.101356 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-eac46fbd-7d96-44bf-ac01-ee49ebae2ec1 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:51:17.102550 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-eac46fbd-7d96-44bf-ac01-ee49ebae2ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:17.111486 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:51:17.161433 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1367/2781] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:51:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:51:17.171919 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869921fa-a529-46a8-9975-f954b7bd31a8 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:17.173311 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-869921fa-a529-46a8-9975-f954b7bd31a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:17.229251 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869921fa-a529-46a8-9975-f954b7bd31a8 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:17.232712 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b0044-0d20-4515-91d8-83d56f0161d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:17.233350 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b0044-0d20-4515-91d8-83d56f0161d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:17.233629 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b0044-0d20-4515-91d8-83d56f0161d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:17.233902 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b0044-0d20-4515-91d8-83d56f0161d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:17.242843 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b0044-0d20-4515-91d8-83d56f0161d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:17.244286 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b0044-0d20-4515-91d8-83d56f0161d3 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:17.253634 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1415/2782] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:51:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:51:17.647109 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac46fbd-7d96-44bf-ac01-ee49ebae2ec1 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:17.656704 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-eac46fbd-7d96-44bf-ac01-ee49ebae2ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:17.661005 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac46fbd-7d96-44bf-ac01-ee49ebae2ec1 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:17.667046 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7d833a-3629-42e5-8fec-0b965e212144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:17.667967 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d833a-3629-42e5-8fec-0b965e212144 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:17.668193 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d833a-3629-42e5-8fec-0b965e212144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:17.668854 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7d833a-3629-42e5-8fec-0b965e212144 admin admin] PATH_INFO: `/v3/projects/3b9fcd5b09b9443f9a00e3a7dc85a10a` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:17.669749 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7d833a-3629-42e5-8fec-0b965e212144 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b9fcd5b09b9443f9a00e3a7dc85a10a)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:17.671663 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7d833a-3629-42e5-8fec-0b965e212144 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:17.773407 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-8d7d833a-3629-42e5-8fec-0b965e212144 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b9fcd5b09b9443f9a00e3a7dc85a10a', 'request_id': 'req-8d7d833a-3629-42e5-8fec-0b965e212144'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:17.780489 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-8d7d833a-3629-42e5-8fec-0b965e212144 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b9fcd5b09b9443f9a00e3a7dc85a10a', 'request_id': 'req-8d7d833a-3629-42e5-8fec-0b965e212144'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:17.828643 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-8d7d833a-3629-42e5-8fec-0b965e212144 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b9fcd5b09b9443f9a00e3a7dc85a10a', 'request_id': 'req-8d7d833a-3629-42e5-8fec-0b965e212144'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:17.839973 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-8d7d833a-3629-42e5-8fec-0b965e212144 admin admin] The token cache is being invalidate because project 3b9fcd5b09b9443f9a00e3a7dc85a10a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:51:17.840383 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-8d7d833a-3629-42e5-8fec-0b965e212144 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:17.841047 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-8d7d833a-3629-42e5-8fec-0b965e212144 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:17.842640 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1368/2783] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:51:17 2022] DELETE /identity/v3/projects/3b9fcd5b09b9443f9a00e3a7dc85a10a => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:51:17.846737 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b0044-0d20-4515-91d8-83d56f0161d3 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:17.856984 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b0044-0d20-4515-91d8-83d56f0161d3 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:17.913308 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d8d189-9adc-48cf-b304-4c226f3f29f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:17.915988 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d8d189-9adc-48cf-b304-4c226f3f29f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:17.915988 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d8d189-9adc-48cf-b304-4c226f3f29f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:17.915988 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d8d189-9adc-48cf-b304-4c226f3f29f6 admin admin] PATH_INFO: `/v3/users/012e850179da48dbb13d2c2728172d88` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:17.916313 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d8d189-9adc-48cf-b304-4c226f3f29f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=012e850179da48dbb13d2c2728172d88)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:17.918064 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d8d189-9adc-48cf-b304-4c226f3f29f6 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:18.058685 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-d5d8d189-9adc-48cf-b304-4c226f3f29f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '012e850179da48dbb13d2c2728172d88'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:18.065327 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-d5d8d189-9adc-48cf-b304-4c226f3f29f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '012e850179da48dbb13d2c2728172d88'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:18.065986 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-d5d8d189-9adc-48cf-b304-4c226f3f29f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '012e850179da48dbb13d2c2728172d88'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:18.081721 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-d5d8d189-9adc-48cf-b304-4c226f3f29f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '012e850179da48dbb13d2c2728172d88'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:18.086073 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:51:18.160198 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-d5d8d189-9adc-48cf-b304-4c226f3f29f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '012e850179da48dbb13d2c2728172d88'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:18.167476 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1416/2784] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:51:17 2022] DELETE /identity/v3/users/012e850179da48dbb13d2c2728172d88 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:51:18.561298 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7d833a-3629-42e5-8fec-0b965e212144 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:18.566466 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-8d7d833a-3629-42e5-8fec-0b965e212144 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:18.576576 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7d833a-3629-42e5-8fec-0b965e212144 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:18.634137 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c5e10-19b8-4397-bddb-b1e77012893c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:18.635004 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c5e10-19b8-4397-bddb-b1e77012893c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:18.635284 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c5e10-19b8-4397-bddb-b1e77012893c admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:18.635558 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c5e10-19b8-4397-bddb-b1e77012893c admin admin] PATH_INFO: `/v3/projects/5f3178904ad2418b9aea21cb31d8dd56` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:18.636775 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175c5e10-19b8-4397-bddb-b1e77012893c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f3178904ad2418b9aea21cb31d8dd56)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:18.638689 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175c5e10-19b8-4397-bddb-b1e77012893c admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:18.712265 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:51:18.718390 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-175c5e10-19b8-4397-bddb-b1e77012893c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f3178904ad2418b9aea21cb31d8dd56', 'request_id': 'req-175c5e10-19b8-4397-bddb-b1e77012893c'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:18.724734 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-175c5e10-19b8-4397-bddb-b1e77012893c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f3178904ad2418b9aea21cb31d8dd56', 'request_id': 'req-175c5e10-19b8-4397-bddb-b1e77012893c'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:18.782809 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-175c5e10-19b8-4397-bddb-b1e77012893c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f3178904ad2418b9aea21cb31d8dd56', 'request_id': 'req-175c5e10-19b8-4397-bddb-b1e77012893c'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:18.794547 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-175c5e10-19b8-4397-bddb-b1e77012893c admin admin] The token cache is being invalidate because project 5f3178904ad2418b9aea21cb31d8dd56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:51:18.799037 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-175c5e10-19b8-4397-bddb-b1e77012893c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:18.799037 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-175c5e10-19b8-4397-bddb-b1e77012893c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:18.799037 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1369/2785] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:51:18 2022] DELETE /identity/v3/projects/5f3178904ad2418b9aea21cb31d8dd56 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:51:18.801918 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d9b2b0-d181-4f26-a8e9-f8daaa10f09b None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:18.802210 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d9b2b0-d181-4f26-a8e9-f8daaa10f09b None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:18.803026 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d9b2b0-d181-4f26-a8e9-f8daaa10f09b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:18.829554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-b5d9b2b0-d181-4f26-a8e9-f8daaa10f09b None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:51:18.829554 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b5d9b2b0-d181-4f26-a8e9-f8daaa10f09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:18.878938 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1417/2786] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:51:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:51:18.887141 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff3d9c-c724-43e0-a03d-309b71d657cd None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:18.887435 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff3d9c-c724-43e0-a03d-309b71d657cd None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:18.887721 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff3d9c-c724-43e0-a03d-309b71d657cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:18.905549 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-57ff3d9c-c724-43e0-a03d-309b71d657cd None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:51:18.906751 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-57ff3d9c-c724-43e0-a03d-309b71d657cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:18.914613 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1370/2787] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:51:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:51:18.918856 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d9b2b0-d181-4f26-a8e9-f8daaa10f09b None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:18.928966 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d9b2b0-d181-4f26-a8e9-f8daaa10f09b None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:18.933510 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d55418-2df8-4b1c-82f0-0ded275f09cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:18.934152 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d55418-2df8-4b1c-82f0-0ded275f09cc admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:18.934434 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d55418-2df8-4b1c-82f0-0ded275f09cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:18.934712 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d55418-2df8-4b1c-82f0-0ded275f09cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:18.935435 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d55418-2df8-4b1c-82f0-0ded275f09cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:51:18.935766 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d55418-2df8-4b1c-82f0-0ded275f09cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:18.937347 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d55418-2df8-4b1c-82f0-0ded275f09cc admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:18.949186 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1418/2788] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:51:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:51:18.953290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ff3d9c-c724-43e0-a03d-309b71d657cd None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:18.962454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ff3d9c-c724-43e0-a03d-309b71d657cd None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:18.967146 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e957752-4e0a-43e2-b69d-e0c7a0b70c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:18.967825 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e957752-4e0a-43e2-b69d-e0c7a0b70c79 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:18.968114 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e957752-4e0a-43e2-b69d-e0c7a0b70c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:18.968394 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e957752-4e0a-43e2-b69d-e0c7a0b70c79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:18.969094 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e957752-4e0a-43e2-b69d-e0c7a0b70c79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:18.970983 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e957752-4e0a-43e2-b69d-e0c7a0b70c79 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:19.093003 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1371/2789] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:51:18 2022] POST /identity/v3/projects => generated 369 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:51:19.097151 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d55418-2df8-4b1c-82f0-0ded275f09cc admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:19.107681 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d55418-2df8-4b1c-82f0-0ded275f09cc admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:19.166812 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea7620-2700-45bb-9d11-a1ca94b7e391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:19.167642 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea7620-2700-45bb-9d11-a1ca94b7e391 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:19.168181 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea7620-2700-45bb-9d11-a1ca94b7e391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:19.168491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea7620-2700-45bb-9d11-a1ca94b7e391 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:19.169819 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fea7620-2700-45bb-9d11-a1ca94b7e391 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:19.171418 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fea7620-2700-45bb-9d11-a1ca94b7e391 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:19.269484 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1419/2790] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:51:19 2022] POST /identity/v3/users => generated 285 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:51:19.273715 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e957752-4e0a-43e2-b69d-e0c7a0b70c79 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:19.284050 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e957752-4e0a-43e2-b69d-e0c7a0b70c79 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:19.289134 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417eb905-befd-4474-80ff-c237063878e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:19.289789 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417eb905-befd-4474-80ff-c237063878e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:19.290077 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417eb905-befd-4474-80ff-c237063878e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:19.290409 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417eb905-befd-4474-80ff-c237063878e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:19.291373 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417eb905-befd-4474-80ff-c237063878e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:19.293001 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417eb905-befd-4474-80ff-c237063878e4 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:19.303820 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1372/2791] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:51:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:51:19.307613 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea7620-2700-45bb-9d11-a1ca94b7e391 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:19.317446 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea7620-2700-45bb-9d11-a1ca94b7e391 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:19.322116 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78225107-0915-4f23-bcc1-b017d995259f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:19.322780 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78225107-0915-4f23-bcc1-b017d995259f admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:19.323062 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78225107-0915-4f23-bcc1-b017d995259f admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:19.323338 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78225107-0915-4f23-bcc1-b017d995259f admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:19.324112 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78225107-0915-4f23-bcc1-b017d995259f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:19.325682 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78225107-0915-4f23-bcc1-b017d995259f admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:19.335463 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1420/2792] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:51:19 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:51:19.339261 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417eb905-befd-4474-80ff-c237063878e4 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:19.349137 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417eb905-befd-4474-80ff-c237063878e4 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:19.354117 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b9844f-1957-4442-9e05-230c47295715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:19.354980 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b9844f-1957-4442-9e05-230c47295715 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:19.355262 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b9844f-1957-4442-9e05-230c47295715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:19.355535 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b9844f-1957-4442-9e05-230c47295715 admin admin] PATH_INFO: `/v3/projects/55e1a3a306de4b00a03723c5edc9c6cd/users/10bee567123a4c45a5ec5c6ab974d709/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:19.371363 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b9844f-1957-4442-9e05-230c47295715 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55e1a3a306de4b00a03723c5edc9c6cd, user_id=10bee567123a4c45a5ec5c6ab974d709, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:19.374019 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b9844f-1957-4442-9e05-230c47295715 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:19.447099 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1373/2793] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:51:19 2022] PUT /identity/v3/projects/55e1a3a306de4b00a03723c5edc9c6cd/users/10bee567123a4c45a5ec5c6ab974d709/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:51:19.458482 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf631e2-8cc3-40e2-97c0-d5f822ebfc37 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:19.459282 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf631e2-8cc3-40e2-97c0-d5f822ebfc37 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:19.459560 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf631e2-8cc3-40e2-97c0-d5f822ebfc37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:19.487089 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-1cf631e2-8cc3-40e2-97c0-d5f822ebfc37 None None] MFA Rules not processed for user `10bee567123a4c45a5ec5c6ab974d709`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:51:19.488336 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-1cf631e2-8cc3-40e2-97c0-d5f822ebfc37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:19.533626 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:51:19.551852 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1421/2794] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:51:19 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:51:19.556363 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b9844f-1957-4442-9e05-230c47295715 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:19.616989 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b9844f-1957-4442-9e05-230c47295715 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:19.621715 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c299a-d368-432e-ab4a-8321fe6f20a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:19.622401 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c299a-d368-432e-ab4a-8321fe6f20a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:19.622690 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c299a-d368-432e-ab4a-8321fe6f20a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:19.622968 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c299a-d368-432e-ab4a-8321fe6f20a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:19.623661 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680c299a-d368-432e-ab4a-8321fe6f20a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:19.625629 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680c299a-d368-432e-ab4a-8321fe6f20a9 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:19.704222 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1374/2795] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:51:19 2022] POST /identity/v3/projects => generated 367 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:51:19.708337 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf631e2-8cc3-40e2-97c0-d5f822ebfc37 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:19.719324 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf631e2-8cc3-40e2-97c0-d5f822ebfc37 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:19.774908 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d903b11-fdd4-4152-abb0-c62b536f7460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:19.775579 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d903b11-fdd4-4152-abb0-c62b536f7460 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:19.775895 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d903b11-fdd4-4152-abb0-c62b536f7460 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:19.776177 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d903b11-fdd4-4152-abb0-c62b536f7460 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:19.776859 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d903b11-fdd4-4152-abb0-c62b536f7460 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:19.778769 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d903b11-fdd4-4152-abb0-c62b536f7460 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:19.874260 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1422/2796] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:51:19 2022] POST /identity/v3/users => generated 290 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:51:19.878200 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c299a-d368-432e-ab4a-8321fe6f20a9 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:19.888985 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c299a-d368-432e-ab4a-8321fe6f20a9 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:19.893483 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d2f865-1cc8-4ae0-a9ea-3e696b263f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:19.895822 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2f865-1cc8-4ae0-a9ea-3e696b263f0f admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:19.895822 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2f865-1cc8-4ae0-a9ea-3e696b263f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:19.895822 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d2f865-1cc8-4ae0-a9ea-3e696b263f0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:19.895822 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d2f865-1cc8-4ae0-a9ea-3e696b263f0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:19.897746 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d2f865-1cc8-4ae0-a9ea-3e696b263f0f admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:19.908252 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1375/2797] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:51:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:51:19.912735 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d903b11-fdd4-4152-abb0-c62b536f7460 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:19.923340 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d903b11-fdd4-4152-abb0-c62b536f7460 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:19.928559 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b87756-7c7e-4b87-a5ba-691f005e2423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:19.929435 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b87756-7c7e-4b87-a5ba-691f005e2423 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:19.929746 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b87756-7c7e-4b87-a5ba-691f005e2423 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:19.930024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b87756-7c7e-4b87-a5ba-691f005e2423 admin admin] PATH_INFO: `/v3/projects/e3e023e3e8384072890762c49d3f7df8/users/7c47c86626254baa93a20ae006b1d8b4/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:19.944213 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b87756-7c7e-4b87-a5ba-691f005e2423 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3e023e3e8384072890762c49d3f7df8, user_id=7c47c86626254baa93a20ae006b1d8b4, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:19.946873 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b87756-7c7e-4b87-a5ba-691f005e2423 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:20.005848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1423/2798] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:51:19 2022] PUT /identity/v3/projects/e3e023e3e8384072890762c49d3f7df8/users/7c47c86626254baa93a20ae006b1d8b4/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:51:20.009777 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d2f865-1cc8-4ae0-a9ea-3e696b263f0f admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:20.019552 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d2f865-1cc8-4ae0-a9ea-3e696b263f0f admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:20.057473 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:51:20.075696 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8422562-286d-4804-b6bd-cb8127d2af57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:20.076451 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8422562-286d-4804-b6bd-cb8127d2af57 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:20.076737 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8422562-286d-4804-b6bd-cb8127d2af57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:20.077015 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8422562-286d-4804-b6bd-cb8127d2af57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:20.077742 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8422562-286d-4804-b6bd-cb8127d2af57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:20.079394 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8422562-286d-4804-b6bd-cb8127d2af57 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:20.088933 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1376/2799] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:51:20 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:51:20.093143 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b87756-7c7e-4b87-a5ba-691f005e2423 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:20.103917 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b87756-7c7e-4b87-a5ba-691f005e2423 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:20.108465 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d017bd2d-feb0-4eb6-a780-22e8d4587b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:20.110624 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017bd2d-feb0-4eb6-a780-22e8d4587b92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:20.110624 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017bd2d-feb0-4eb6-a780-22e8d4587b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:20.110624 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017bd2d-feb0-4eb6-a780-22e8d4587b92 admin admin] PATH_INFO: `/v3/projects/e3e023e3e8384072890762c49d3f7df8/users/7c47c86626254baa93a20ae006b1d8b4/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:20.111959 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d017bd2d-feb0-4eb6-a780-22e8d4587b92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3e023e3e8384072890762c49d3f7df8, user_id=7c47c86626254baa93a20ae006b1d8b4, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:20.114574 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d017bd2d-feb0-4eb6-a780-22e8d4587b92 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:20.125282 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1424/2800] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:51:20 2022] PUT /identity/v3/projects/e3e023e3e8384072890762c49d3f7df8/users/7c47c86626254baa93a20ae006b1d8b4/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:51:20.135037 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dde114-35c8-4581-aa27-ef6e61ce03ce None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:20.135327 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dde114-35c8-4581-aa27-ef6e61ce03ce None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:20.135607 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dde114-35c8-4581-aa27-ef6e61ce03ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:20.161326 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-c2dde114-35c8-4581-aa27-ef6e61ce03ce None None] MFA Rules not processed for user `7c47c86626254baa93a20ae006b1d8b4`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:51:20.162565 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c2dde114-35c8-4581-aa27-ef6e61ce03ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:20.235487 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1377/2801] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:51:20 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:51:20.254564 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d017bd2d-feb0-4eb6-a780-22e8d4587b92 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:20.256328 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d017bd2d-feb0-4eb6-a780-22e8d4587b92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:20.313477 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d017bd2d-feb0-4eb6-a780-22e8d4587b92 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:20.317149 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3739e6-7d75-4ced-ab55-ba69994d7e82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:20.317790 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3739e6-7d75-4ced-ab55-ba69994d7e82 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:20.318075 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3739e6-7d75-4ced-ab55-ba69994d7e82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:20.318351 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3739e6-7d75-4ced-ab55-ba69994d7e82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:20.327276 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3739e6-7d75-4ced-ab55-ba69994d7e82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:20.328766 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3739e6-7d75-4ced-ab55-ba69994d7e82 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:20.338646 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1425/2802] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:51:20 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:51:20.442105 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dde114-35c8-4581-aa27-ef6e61ce03ce None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:20.451561 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dde114-35c8-4581-aa27-ef6e61ce03ce None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:20.455146 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147b778-04aa-4577-a152-c39469fbd939 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:20.455812 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147b778-04aa-4577-a152-c39469fbd939 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:20.456097 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147b778-04aa-4577-a152-c39469fbd939 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:20.456372 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147b778-04aa-4577-a152-c39469fbd939 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:20.512388 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e147b778-04aa-4577-a152-c39469fbd939 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:20.513912 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e147b778-04aa-4577-a152-c39469fbd939 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:20.523287 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1378/2803] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:51:20 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:51:24.956095 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3739e6-7d75-4ced-ab55-ba69994d7e82 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:25.000835 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:51:25.016736 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3739e6-7d75-4ced-ab55-ba69994d7e82 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:25.021252 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a804ed-f19b-4e8d-91ce-b97bd924d385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:25.022123 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a804ed-f19b-4e8d-91ce-b97bd924d385 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:25.022411 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a804ed-f19b-4e8d-91ce-b97bd924d385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:25.022684 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a804ed-f19b-4e8d-91ce-b97bd924d385 admin admin] PATH_INFO: `/v3/users/10bee567123a4c45a5ec5c6ab974d709` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:25.023911 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a804ed-f19b-4e8d-91ce-b97bd924d385 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10bee567123a4c45a5ec5c6ab974d709)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:25.025795 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a804ed-f19b-4e8d-91ce-b97bd924d385 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:25.254248 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-09a804ed-f19b-4e8d-91ce-b97bd924d385 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10bee567123a4c45a5ec5c6ab974d709'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:25.261001 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-09a804ed-f19b-4e8d-91ce-b97bd924d385 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10bee567123a4c45a5ec5c6ab974d709'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:25.261772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-09a804ed-f19b-4e8d-91ce-b97bd924d385 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10bee567123a4c45a5ec5c6ab974d709'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:25.273362 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-09a804ed-f19b-4e8d-91ce-b97bd924d385 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10bee567123a4c45a5ec5c6ab974d709'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:25.337052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-09a804ed-f19b-4e8d-91ce-b97bd924d385 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10bee567123a4c45a5ec5c6ab974d709'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:25.343081 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1426/2804] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:51:24 2022] DELETE /identity/v3/users/10bee567123a4c45a5ec5c6ab974d709 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:51:25.347606 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc03d94-738c-4518-81c4-2660a533c46d None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:25.348149 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc03d94-738c-4518-81c4-2660a533c46d None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:25.348299 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc03d94-738c-4518-81c4-2660a533c46d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:25.365722 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-cfc03d94-738c-4518-81c4-2660a533c46d None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:51:25.367526 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-cfc03d94-738c-4518-81c4-2660a533c46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:25.416835 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=100 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:51:25.424860 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1379/2805] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:51:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:51:25.435642 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a804ed-f19b-4e8d-91ce-b97bd924d385 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:25.437060 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-09a804ed-f19b-4e8d-91ce-b97bd924d385 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:25.493280 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a804ed-f19b-4e8d-91ce-b97bd924d385 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:25.496782 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5e48d-ace7-494e-a9ed-7c0b10ebc599 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:25.497417 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5e48d-ace7-494e-a9ed-7c0b10ebc599 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:25.497695 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5e48d-ace7-494e-a9ed-7c0b10ebc599 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:25.497967 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5e48d-ace7-494e-a9ed-7c0b10ebc599 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:25.506771 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d5e48d-ace7-494e-a9ed-7c0b10ebc599 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:25.508156 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d5e48d-ace7-494e-a9ed-7c0b10ebc599 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:25.517661 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1427/2806] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:51:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:51:25.906019 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc03d94-738c-4518-81c4-2660a533c46d None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:25.907459 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-cfc03d94-738c-4518-81c4-2660a533c46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:25.920255 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc03d94-738c-4518-81c4-2660a533c46d None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:25.925497 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0451e9a5-8c32-4dd7-8467-80a620e716ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:25.926374 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451e9a5-8c32-4dd7-8467-80a620e716ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:25.926685 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451e9a5-8c32-4dd7-8467-80a620e716ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:25.926964 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451e9a5-8c32-4dd7-8467-80a620e716ee admin admin] PATH_INFO: `/v3/projects/55e1a3a306de4b00a03723c5edc9c6cd` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:25.928189 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0451e9a5-8c32-4dd7-8467-80a620e716ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=55e1a3a306de4b00a03723c5edc9c6cd)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:25.930162 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0451e9a5-8c32-4dd7-8467-80a620e716ee admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:26.041725 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0451e9a5-8c32-4dd7-8467-80a620e716ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55e1a3a306de4b00a03723c5edc9c6cd', 'request_id': 'req-0451e9a5-8c32-4dd7-8467-80a620e716ee'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:26.047893 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0451e9a5-8c32-4dd7-8467-80a620e716ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55e1a3a306de4b00a03723c5edc9c6cd', 'request_id': 'req-0451e9a5-8c32-4dd7-8467-80a620e716ee'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:26.137798 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0451e9a5-8c32-4dd7-8467-80a620e716ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55e1a3a306de4b00a03723c5edc9c6cd', 'request_id': 'req-0451e9a5-8c32-4dd7-8467-80a620e716ee'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:26.148942 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0451e9a5-8c32-4dd7-8467-80a620e716ee admin admin] The token cache is being invalidate because project 55e1a3a306de4b00a03723c5edc9c6cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:51:26.149231 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0451e9a5-8c32-4dd7-8467-80a620e716ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:26.149849 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-0451e9a5-8c32-4dd7-8467-80a620e716ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:26.151445 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1380/2807] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:51:25 2022] DELETE /identity/v3/projects/55e1a3a306de4b00a03723c5edc9c6cd => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:51:26.155507 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5e48d-ace7-494e-a9ed-7c0b10ebc599 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:26.164996 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5e48d-ace7-494e-a9ed-7c0b10ebc599 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:26.226127 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283af9ee-79aa-4a28-8e9f-3bb2e77ece7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:26.227005 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283af9ee-79aa-4a28-8e9f-3bb2e77ece7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:26.227290 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283af9ee-79aa-4a28-8e9f-3bb2e77ece7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:26.227590 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283af9ee-79aa-4a28-8e9f-3bb2e77ece7d admin admin] PATH_INFO: `/v3/users/7c47c86626254baa93a20ae006b1d8b4` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:26.228821 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283af9ee-79aa-4a28-8e9f-3bb2e77ece7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c47c86626254baa93a20ae006b1d8b4)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:26.230722 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283af9ee-79aa-4a28-8e9f-3bb2e77ece7d admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:26.431287 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:51:26.438665 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-283af9ee-79aa-4a28-8e9f-3bb2e77ece7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c47c86626254baa93a20ae006b1d8b4'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:26.447476 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-283af9ee-79aa-4a28-8e9f-3bb2e77ece7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c47c86626254baa93a20ae006b1d8b4'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:26.448205 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-283af9ee-79aa-4a28-8e9f-3bb2e77ece7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c47c86626254baa93a20ae006b1d8b4'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:26.460421 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-283af9ee-79aa-4a28-8e9f-3bb2e77ece7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c47c86626254baa93a20ae006b1d8b4'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:26.606240 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-283af9ee-79aa-4a28-8e9f-3bb2e77ece7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c47c86626254baa93a20ae006b1d8b4'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:26.612530 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1428/2808] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:51:26 2022] DELETE /identity/v3/users/7c47c86626254baa93a20ae006b1d8b4 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:51:27.053597 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0451e9a5-8c32-4dd7-8467-80a620e716ee admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:27.055687 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-0451e9a5-8c32-4dd7-8467-80a620e716ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:27.071421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0451e9a5-8c32-4dd7-8467-80a620e716ee admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:27.128876 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2e0c4d-20f5-46d9-9926-0b08f60b1c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:27.129748 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e0c4d-20f5-46d9-9926-0b08f60b1c49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:27.130058 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e0c4d-20f5-46d9-9926-0b08f60b1c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:27.130335 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2e0c4d-20f5-46d9-9926-0b08f60b1c49 admin admin] PATH_INFO: `/v3/projects/e3e023e3e8384072890762c49d3f7df8` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:27.131529 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2e0c4d-20f5-46d9-9926-0b08f60b1c49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3e023e3e8384072890762c49d3f7df8)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:27.134112 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2e0c4d-20f5-46d9-9926-0b08f60b1c49 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:27.298087 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6b2e0c4d-20f5-46d9-9926-0b08f60b1c49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3e023e3e8384072890762c49d3f7df8', 'request_id': 'req-6b2e0c4d-20f5-46d9-9926-0b08f60b1c49'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:27.304889 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6b2e0c4d-20f5-46d9-9926-0b08f60b1c49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3e023e3e8384072890762c49d3f7df8', 'request_id': 'req-6b2e0c4d-20f5-46d9-9926-0b08f60b1c49'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:27.400092 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6b2e0c4d-20f5-46d9-9926-0b08f60b1c49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3e023e3e8384072890762c49d3f7df8', 'request_id': 'req-6b2e0c4d-20f5-46d9-9926-0b08f60b1c49'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:27.410384 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6b2e0c4d-20f5-46d9-9926-0b08f60b1c49 admin admin] The token cache is being invalidate because project e3e023e3e8384072890762c49d3f7df8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:51:27.410672 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6b2e0c4d-20f5-46d9-9926-0b08f60b1c49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:27.411294 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6b2e0c4d-20f5-46d9-9926-0b08f60b1c49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:27.412855 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1381/2809] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:51:27 2022] DELETE /identity/v3/projects/e3e023e3e8384072890762c49d3f7df8 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:51:27.417995 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe066a-e559-497d-80e6-c3e9c3cc7ac3 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:27.418435 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe066a-e559-497d-80e6-c3e9c3cc7ac3 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:27.418560 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe066a-e559-497d-80e6-c3e9c3cc7ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:27.436525 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-9cbe066a-e559-497d-80e6-c3e9c3cc7ac3 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:51:27.437729 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-9cbe066a-e559-497d-80e6-c3e9c3cc7ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:27.495530 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1429/2810] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:51:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:51:27.504398 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f8216-6903-4bab-b693-11482536ebdf None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:27.504813 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f8216-6903-4bab-b693-11482536ebdf None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:27.505102 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f8216-6903-4bab-b693-11482536ebdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:27.525550 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-676f8216-6903-4bab-b693-11482536ebdf None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:51:27.526751 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-676f8216-6903-4bab-b693-11482536ebdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:27.534631 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1382/2811] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:51:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:51:27.538695 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe066a-e559-497d-80e6-c3e9c3cc7ac3 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:27.548444 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe066a-e559-497d-80e6-c3e9c3cc7ac3 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:27.552925 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdefcf5-bb11-4f9e-ae94-e5c38ad22f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:27.553574 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdefcf5-bb11-4f9e-ae94-e5c38ad22f13 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:27.553857 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdefcf5-bb11-4f9e-ae94-e5c38ad22f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:27.554131 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdefcf5-bb11-4f9e-ae94-e5c38ad22f13 admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:27.554860 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdefcf5-bb11-4f9e-ae94-e5c38ad22f13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:51:27.555170 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdefcf5-bb11-4f9e-ae94-e5c38ad22f13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:27.556779 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdefcf5-bb11-4f9e-ae94-e5c38ad22f13 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:27.568052 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1430/2812] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:51:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:51:27.572720 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676f8216-6903-4bab-b693-11482536ebdf None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:27.581699 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676f8216-6903-4bab-b693-11482536ebdf None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:27.586894 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70edc941-456c-47c1-89f1-ef3d22a06124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:27.587529 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70edc941-456c-47c1-89f1-ef3d22a06124 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:27.587914 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70edc941-456c-47c1-89f1-ef3d22a06124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:27.588219 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70edc941-456c-47c1-89f1-ef3d22a06124 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:27.588908 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70edc941-456c-47c1-89f1-ef3d22a06124 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:27.590777 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70edc941-456c-47c1-89f1-ef3d22a06124 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:27.593920 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:51:27.670487 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1383/2813] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:51:27 2022] POST /identity/v3/projects => generated 371 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:51:27.674525 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdefcf5-bb11-4f9e-ae94-e5c38ad22f13 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:27.684897 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdefcf5-bb11-4f9e-ae94-e5c38ad22f13 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:27.743376 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e378b78-bbfb-459c-aeb6-2a00d9cdf95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:27.744080 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e378b78-bbfb-459c-aeb6-2a00d9cdf95f admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:27.744363 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e378b78-bbfb-459c-aeb6-2a00d9cdf95f admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:27.744638 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e378b78-bbfb-459c-aeb6-2a00d9cdf95f admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:27.745322 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e378b78-bbfb-459c-aeb6-2a00d9cdf95f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:27.747216 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e378b78-bbfb-459c-aeb6-2a00d9cdf95f admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:27.823709 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1431/2814] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:51:27 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:51:27.827765 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70edc941-456c-47c1-89f1-ef3d22a06124 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:27.838288 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70edc941-456c-47c1-89f1-ef3d22a06124 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:27.843002 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d716cbe-f21c-4210-af0f-f93fd3a46416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:27.843651 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d716cbe-f21c-4210-af0f-f93fd3a46416 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:27.844045 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d716cbe-f21c-4210-af0f-f93fd3a46416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:27.844323 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d716cbe-f21c-4210-af0f-f93fd3a46416 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:27.845039 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d716cbe-f21c-4210-af0f-f93fd3a46416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:27.846622 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d716cbe-f21c-4210-af0f-f93fd3a46416 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:27.857038 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1384/2815] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:51:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:51:27.860668 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e378b78-bbfb-459c-aeb6-2a00d9cdf95f admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:27.865610 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:51:27.871242 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e378b78-bbfb-459c-aeb6-2a00d9cdf95f admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:27.875910 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6376463-6c5e-4334-9232-63a57b5e6b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:27.876662 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6376463-6c5e-4334-9232-63a57b5e6b38 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:27.876946 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6376463-6c5e-4334-9232-63a57b5e6b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:27.877223 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6376463-6c5e-4334-9232-63a57b5e6b38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:27.877994 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6376463-6c5e-4334-9232-63a57b5e6b38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:27.879559 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6376463-6c5e-4334-9232-63a57b5e6b38 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:27.890439 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1432/2816] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:51:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:51:27.894336 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d716cbe-f21c-4210-af0f-f93fd3a46416 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:27.904312 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d716cbe-f21c-4210-af0f-f93fd3a46416 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:27.909361 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b882b-8529-4c0a-bcc7-4a1a365925b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:27.910315 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b882b-8529-4c0a-bcc7-4a1a365925b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:27.910689 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b882b-8529-4c0a-bcc7-4a1a365925b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:27.911031 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5b882b-8529-4c0a-bcc7-4a1a365925b8 admin admin] PATH_INFO: `/v3/projects/fa4e56b21b894a5288b7c13eaab65f28/users/24cc528e66604ae4b04137363eafe2b1/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:27.925893 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5b882b-8529-4c0a-bcc7-4a1a365925b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa4e56b21b894a5288b7c13eaab65f28, user_id=24cc528e66604ae4b04137363eafe2b1, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:27.928556 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5b882b-8529-4c0a-bcc7-4a1a365925b8 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:28.017515 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1385/2817] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:51:27 2022] PUT /identity/v3/projects/fa4e56b21b894a5288b7c13eaab65f28/users/24cc528e66604ae4b04137363eafe2b1/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:51:28.027847 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc59cff4-f9fa-40b1-8ba1-7e3574820515 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:28.028082 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc59cff4-f9fa-40b1-8ba1-7e3574820515 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:28.028362 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc59cff4-f9fa-40b1-8ba1-7e3574820515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:28.054882 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-cc59cff4-f9fa-40b1-8ba1-7e3574820515 None None] MFA Rules not processed for user `24cc528e66604ae4b04137363eafe2b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:51:28.056094 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-cc59cff4-f9fa-40b1-8ba1-7e3574820515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:28.118291 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1433/2818] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:51:28 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:51:28.129246 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b882b-8529-4c0a-bcc7-4a1a365925b8 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:28.131051 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4f5b882b-8529-4c0a-bcc7-4a1a365925b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:28.186226 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5b882b-8529-4c0a-bcc7-4a1a365925b8 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:28.189678 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0b3074-82da-4ecd-a927-b6e07cc3cf59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:28.190313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b3074-82da-4ecd-a927-b6e07cc3cf59 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:28.190596 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b3074-82da-4ecd-a927-b6e07cc3cf59 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:28.190872 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0b3074-82da-4ecd-a927-b6e07cc3cf59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:28.199053 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0b3074-82da-4ecd-a927-b6e07cc3cf59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:28.200520 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0b3074-82da-4ecd-a927-b6e07cc3cf59 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:28.209052 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1386/2819] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:51:28 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:51:28.228865 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc59cff4-f9fa-40b1-8ba1-7e3574820515 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:28.295903 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc59cff4-f9fa-40b1-8ba1-7e3574820515 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:28.300473 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71451d9-a075-4b83-8281-920680086a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:28.301341 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71451d9-a075-4b83-8281-920680086a89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:28.301652 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71451d9-a075-4b83-8281-920680086a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:28.301929 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71451d9-a075-4b83-8281-920680086a89 admin admin] PATH_INFO: `/v3/users/24cc528e66604ae4b04137363eafe2b1` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:28.303128 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71451d9-a075-4b83-8281-920680086a89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24cc528e66604ae4b04137363eafe2b1)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:28.305698 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71451d9-a075-4b83-8281-920680086a89 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:28.498717 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f71451d9-a075-4b83-8281-920680086a89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24cc528e66604ae4b04137363eafe2b1'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:28.505602 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f71451d9-a075-4b83-8281-920680086a89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24cc528e66604ae4b04137363eafe2b1'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:28.506329 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f71451d9-a075-4b83-8281-920680086a89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24cc528e66604ae4b04137363eafe2b1'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:28.518242 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f71451d9-a075-4b83-8281-920680086a89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24cc528e66604ae4b04137363eafe2b1'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:28.582676 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-f71451d9-a075-4b83-8281-920680086a89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24cc528e66604ae4b04137363eafe2b1'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:28.584791 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:51:28.589587 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1434/2820] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:51:28 2022] DELETE /identity/v3/users/24cc528e66604ae4b04137363eafe2b1 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:51:28.594048 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7840e3cd-42cb-4d2c-a134-90241a08725c None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:28.594329 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7840e3cd-42cb-4d2c-a134-90241a08725c None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:28.594610 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7840e3cd-42cb-4d2c-a134-90241a08725c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:28.612098 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-7840e3cd-42cb-4d2c-a134-90241a08725c None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:51:28.613286 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-7840e3cd-42cb-4d2c-a134-90241a08725c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:28.669994 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1387/2821] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:51:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:51:28.679590 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71451d9-a075-4b83-8281-920680086a89 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:28.681056 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f71451d9-a075-4b83-8281-920680086a89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:28.737652 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71451d9-a075-4b83-8281-920680086a89 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:28.741284 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10670fcf-89e2-4be1-bd68-67e903dd7da5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:28.741929 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10670fcf-89e2-4be1-bd68-67e903dd7da5 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:28.742246 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10670fcf-89e2-4be1-bd68-67e903dd7da5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:28.742526 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10670fcf-89e2-4be1-bd68-67e903dd7da5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:28.751343 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10670fcf-89e2-4be1-bd68-67e903dd7da5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:28.752758 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10670fcf-89e2-4be1-bd68-67e903dd7da5 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:28.762540 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1435/2822] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:51:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:51:29.219434 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7840e3cd-42cb-4d2c-a134-90241a08725c None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:29.220983 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-7840e3cd-42cb-4d2c-a134-90241a08725c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:29.234252 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7840e3cd-42cb-4d2c-a134-90241a08725c None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:29.239003 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49183a5-09b6-4bff-99e0-fcad1cf1f0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:29.240622 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49183a5-09b6-4bff-99e0-fcad1cf1f0e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:29.240912 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49183a5-09b6-4bff-99e0-fcad1cf1f0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:29.241188 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49183a5-09b6-4bff-99e0-fcad1cf1f0e9 admin admin] PATH_INFO: `/v3/projects/fa4e56b21b894a5288b7c13eaab65f28` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:29.242994 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49183a5-09b6-4bff-99e0-fcad1cf1f0e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa4e56b21b894a5288b7c13eaab65f28)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:29.244844 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49183a5-09b6-4bff-99e0-fcad1cf1f0e9 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:29.342541 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:51:29.343774 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-e49183a5-09b6-4bff-99e0-fcad1cf1f0e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa4e56b21b894a5288b7c13eaab65f28', 'request_id': 'req-e49183a5-09b6-4bff-99e0-fcad1cf1f0e9'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:29.349873 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-e49183a5-09b6-4bff-99e0-fcad1cf1f0e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa4e56b21b894a5288b7c13eaab65f28', 'request_id': 'req-e49183a5-09b6-4bff-99e0-fcad1cf1f0e9'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:29.446397 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-e49183a5-09b6-4bff-99e0-fcad1cf1f0e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa4e56b21b894a5288b7c13eaab65f28', 'request_id': 'req-e49183a5-09b6-4bff-99e0-fcad1cf1f0e9'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:29.458131 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-e49183a5-09b6-4bff-99e0-fcad1cf1f0e9 admin admin] The token cache is being invalidate because project fa4e56b21b894a5288b7c13eaab65f28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:51:29.458513 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-e49183a5-09b6-4bff-99e0-fcad1cf1f0e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:29.459156 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-e49183a5-09b6-4bff-99e0-fcad1cf1f0e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:51:29.460816 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1388/2823] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:51:29 2022] DELETE /identity/v3/projects/fa4e56b21b894a5288b7c13eaab65f28 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:51:29.465753 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3914c2-7b0a-47a2-98f3-3fb9ce161169 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:29.466042 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3914c2-7b0a-47a2-98f3-3fb9ce161169 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:29.466363 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3914c2-7b0a-47a2-98f3-3fb9ce161169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:29.485324 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-7e3914c2-7b0a-47a2-98f3-3fb9ce161169 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:51:29.486514 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-7e3914c2-7b0a-47a2-98f3-3fb9ce161169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:29.546121 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1436/2824] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:51:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:51:29.553782 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83acaa-71a5-4126-9c8d-481529137ffb None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:29.554071 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83acaa-71a5-4126-9c8d-481529137ffb None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:29.554344 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b83acaa-71a5-4126-9c8d-481529137ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:29.573605 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-9b83acaa-71a5-4126-9c8d-481529137ffb None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:51:29.574835 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-9b83acaa-71a5-4126-9c8d-481529137ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:29.583016 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1389/2825] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:51:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:51:29.587440 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3914c2-7b0a-47a2-98f3-3fb9ce161169 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:29.597160 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3914c2-7b0a-47a2-98f3-3fb9ce161169 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:29.602233 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8305774-b546-4bd6-875c-6c6643e905d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:29.602887 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8305774-b546-4bd6-875c-6c6643e905d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:29.603170 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8305774-b546-4bd6-875c-6c6643e905d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:29.603448 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8305774-b546-4bd6-875c-6c6643e905d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:29.604214 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8305774-b546-4bd6-875c-6c6643e905d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:51:29.604523 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8305774-b546-4bd6-875c-6c6643e905d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:29.606097 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8305774-b546-4bd6-875c-6c6643e905d4 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:29.618129 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1437/2826] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:51:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:51:29.622239 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83acaa-71a5-4126-9c8d-481529137ffb None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:29.631321 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b83acaa-71a5-4126-9c8d-481529137ffb None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:29.636023 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ec4b7-0724-4be1-9b7c-1ee47815cb4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:29.636664 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ec4b7-0724-4be1-9b7c-1ee47815cb4b admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:29.636947 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ec4b7-0724-4be1-9b7c-1ee47815cb4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:29.637221 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ec4b7-0724-4be1-9b7c-1ee47815cb4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:29.657104 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0ec4b7-0724-4be1-9b7c-1ee47815cb4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:29.657104 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0ec4b7-0724-4be1-9b7c-1ee47815cb4b admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:29.746468 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1390/2827] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:51:29 2022] POST /identity/v3/projects => generated 371 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:51:29.750121 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8305774-b546-4bd6-875c-6c6643e905d4 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:29.760089 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8305774-b546-4bd6-875c-6c6643e905d4 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:29.814792 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d988ea0d-7723-4ad2-97f8-89fbeeb53c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:29.815387 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988ea0d-7723-4ad2-97f8-89fbeeb53c2b admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:29.815682 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988ea0d-7723-4ad2-97f8-89fbeeb53c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:29.815978 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988ea0d-7723-4ad2-97f8-89fbeeb53c2b admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:29.816659 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d988ea0d-7723-4ad2-97f8-89fbeeb53c2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:29.818549 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d988ea0d-7723-4ad2-97f8-89fbeeb53c2b admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:29.885719 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1438/2828] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:51:29 2022] POST /identity/v3/users => generated 286 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:51:29.889710 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ec4b7-0724-4be1-9b7c-1ee47815cb4b admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:29.900676 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ec4b7-0724-4be1-9b7c-1ee47815cb4b admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:29.905224 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2026eb3-539a-45a8-8d2d-1a56f8e4009d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:29.905879 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2026eb3-539a-45a8-8d2d-1a56f8e4009d admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:29.906161 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2026eb3-539a-45a8-8d2d-1a56f8e4009d admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:29.906435 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2026eb3-539a-45a8-8d2d-1a56f8e4009d admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:29.907179 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2026eb3-539a-45a8-8d2d-1a56f8e4009d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:29.908783 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2026eb3-539a-45a8-8d2d-1a56f8e4009d admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:29.920085 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1391/2829] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:51:29.923764 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d988ea0d-7723-4ad2-97f8-89fbeeb53c2b admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:29.933889 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d988ea0d-7723-4ad2-97f8-89fbeeb53c2b admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:29.938481 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0928e0-a90d-4594-87da-3068364b741c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:29.939130 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0928e0-a90d-4594-87da-3068364b741c admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:29.939413 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0928e0-a90d-4594-87da-3068364b741c admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:29.939690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0928e0-a90d-4594-87da-3068364b741c admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:29.940439 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0928e0-a90d-4594-87da-3068364b741c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:29.942022 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0928e0-a90d-4594-87da-3068364b741c admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:29.951943 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1439/2830] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:51:29 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:51:29.955933 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2026eb3-539a-45a8-8d2d-1a56f8e4009d admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:29.966221 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2026eb3-539a-45a8-8d2d-1a56f8e4009d admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:29.970879 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a3a318-a211-4805-a9fa-44c222ca7ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:29.971780 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a3a318-a211-4805-a9fa-44c222ca7ef6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:29.972095 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a3a318-a211-4805-a9fa-44c222ca7ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:29.972376 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a3a318-a211-4805-a9fa-44c222ca7ef6 admin admin] PATH_INFO: `/v3/projects/387788bc8d4e4afab5a34f746f475983/users/f437559c9a344df7bde785d5e494516b/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:29.987240 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a3a318-a211-4805-a9fa-44c222ca7ef6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=387788bc8d4e4afab5a34f746f475983, user_id=f437559c9a344df7bde785d5e494516b, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:29.989900 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a3a318-a211-4805-a9fa-44c222ca7ef6 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:30.081603 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1392/2831] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:51:29 2022] PUT /identity/v3/projects/387788bc8d4e4afab5a34f746f475983/users/f437559c9a344df7bde785d5e494516b/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:51:30.088055 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298b506-b1b5-47c9-a203-c650d2eedad7 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:30.088347 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298b506-b1b5-47c9-a203-c650d2eedad7 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:30.088625 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9298b506-b1b5-47c9-a203-c650d2eedad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:30.106365 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-9298b506-b1b5-47c9-a203-c650d2eedad7 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:51:30.107569 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-9298b506-b1b5-47c9-a203-c650d2eedad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:30.111440 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:51:30.165368 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1440/2832] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:51:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:51:30.175396 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a3a318-a211-4805-a9fa-44c222ca7ef6 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:30.177177 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-29a3a318-a211-4805-a9fa-44c222ca7ef6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:30.232651 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a3a318-a211-4805-a9fa-44c222ca7ef6 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:30.236481 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2456ea9-7a79-4247-9ae5-288463e9579c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:30.237120 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2456ea9-7a79-4247-9ae5-288463e9579c service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:30.237396 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2456ea9-7a79-4247-9ae5-288463e9579c service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:30.237700 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2456ea9-7a79-4247-9ae5-288463e9579c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:30.246555 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2456ea9-7a79-4247-9ae5-288463e9579c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:30.247996 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2456ea9-7a79-4247-9ae5-288463e9579c service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:30.257312 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1393/2833] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:51:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:51:31.398187 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd3c6c-9f01-4489-b433-4114ff3e582b None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:31.398655 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd3c6c-9f01-4489-b433-4114ff3e582b None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:31.399407 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edd3c6c-9f01-4489-b433-4114ff3e582b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:31.434636 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-9edd3c6c-9f01-4489-b433-4114ff3e582b None None] MFA Rules not processed for user `f437559c9a344df7bde785d5e494516b`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:51:31.435878 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-9edd3c6c-9f01-4489-b433-4114ff3e582b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:31.507475 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1441/2834] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:51:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:51:31.520118 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2456ea9-7a79-4247-9ae5-288463e9579c service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:51:31.521671 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-e2456ea9-7a79-4247-9ae5-288463e9579c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:51:31.577871 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2456ea9-7a79-4247-9ae5-288463e9579c service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:51:31.581455 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4624a0ec-9dfe-4759-b7b8-9ee8db75d4b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:51:31.582093 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624a0ec-9dfe-4759-b7b8-9ee8db75d4b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:51:31.582377 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624a0ec-9dfe-4759-b7b8-9ee8db75d4b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:51:31.582650 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4624a0ec-9dfe-4759-b7b8-9ee8db75d4b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:51:31.591340 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4624a0ec-9dfe-4759-b7b8-9ee8db75d4b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:51:31.592804 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4624a0ec-9dfe-4759-b7b8-9ee8db75d4b1 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:51:31.601435 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1394/2835] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:51:31 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:51:41.504464 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=55 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:51:41.597480 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=86,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:52:06.803112 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a469e9-cd32-4479-a411-1a22aa0ef788 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:06.803630 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a469e9-cd32-4479-a411-1a22aa0ef788 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:06.803765 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a469e9-cd32-4479-a411-1a22aa0ef788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:06.820963 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-98a469e9-cd32-4479-a411-1a22aa0ef788 None None] MFA Rules not processed for user `f437559c9a344df7bde785d5e494516b`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:52:06.822664 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-98a469e9-cd32-4479-a411-1a22aa0ef788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:06.829600 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1442/2836] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:52:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:52:07.044258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4624a0ec-9dfe-4759-b7b8-9ee8db75d4b1 service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:07.052877 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4624a0ec-9dfe-4759-b7b8-9ee8db75d4b1 service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:07.057004 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ff307-e803-43e8-8e7d-2c1a0caa77fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:07.057643 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ff307-e803-43e8-8e7d-2c1a0caa77fc service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:07.057932 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ff307-e803-43e8-8e7d-2c1a0caa77fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:07.058211 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260ff307-e803-43e8-8e7d-2c1a0caa77fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:07.060136 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-260ff307-e803-43e8-8e7d-2c1a0caa77fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:07.067998 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a469e9-cd32-4479-a411-1a22aa0ef788 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:07.070327 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-98a469e9-cd32-4479-a411-1a22aa0ef788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:07.081960 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a469e9-cd32-4479-a411-1a22aa0ef788 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:07.085494 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96769e24-25ee-43b0-adc0-aec3e12e7cc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:07.086130 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96769e24-25ee-43b0-adc0-aec3e12e7cc4 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:07.086411 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96769e24-25ee-43b0-adc0-aec3e12e7cc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:07.086685 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96769e24-25ee-43b0-adc0-aec3e12e7cc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:07.088461 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-96769e24-25ee-43b0-adc0-aec3e12e7cc4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:07.097982 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96769e24-25ee-43b0-adc0-aec3e12e7cc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:07.099486 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96769e24-25ee-43b0-adc0-aec3e12e7cc4 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:07.107714 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1443/2837] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:52:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 17:52:07.125067 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260ff307-e803-43e8-8e7d-2c1a0caa77fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:07.126523 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260ff307-e803-43e8-8e7d-2c1a0caa77fc service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:07.137249 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1395/2838] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:52:07 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:52:07.490255 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96769e24-25ee-43b0-adc0-aec3e12e7cc4 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:07.491927 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-96769e24-25ee-43b0-adc0-aec3e12e7cc4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:07.577417 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96769e24-25ee-43b0-adc0-aec3e12e7cc4 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:07.582708 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b6bdc-af29-4b58-a5f2-64cd5374d768 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76ca877bee9041d28ad41e45ef3dd18c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:07.583371 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b6bdc-af29-4b58-a5f2-64cd5374d768 service placement] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:07.583662 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b6bdc-af29-4b58-a5f2-64cd5374d768 service placement] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:07.583975 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5b6bdc-af29-4b58-a5f2-64cd5374d768 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:07.585832 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-3b5b6bdc-af29-4b58-a5f2-64cd5374d768 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:07.600267 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5b6bdc-af29-4b58-a5f2-64cd5374d768 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:07.600267 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5b6bdc-af29-4b58-a5f2-64cd5374d768 service placement] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:07.610185 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1444/2839] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:52:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:52:08.694484 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ff307-e803-43e8-8e7d-2c1a0caa77fc service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:08.704713 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260ff307-e803-43e8-8e7d-2c1a0caa77fc service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:08.708628 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f168212-ae21-4dec-9a43-fad1238452f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:08.709283 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f168212-ae21-4dec-9a43-fad1238452f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:08.709573 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f168212-ae21-4dec-9a43-fad1238452f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:08.709852 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f168212-ae21-4dec-9a43-fad1238452f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:08.712421 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4f168212-ae21-4dec-9a43-fad1238452f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:08.731110 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f168212-ae21-4dec-9a43-fad1238452f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:08.732740 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f168212-ae21-4dec-9a43-fad1238452f2 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:08.744249 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1396/2840] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:52:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:52:09.441467 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b6bdc-af29-4b58-a5f2-64cd5374d768 service placement] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:09.442913 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-3b5b6bdc-af29-4b58-a5f2-64cd5374d768 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:09.456949 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5b6bdc-af29-4b58-a5f2-64cd5374d768 service placement] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:09.462149 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a097d4-b222-464e-ac27-9a236572d04d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:09.463029 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a097d4-b222-464e-ac27-9a236572d04d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:09.463319 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a097d4-b222-464e-ac27-9a236572d04d admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:09.463660 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a097d4-b222-464e-ac27-9a236572d04d admin admin] PATH_INFO: `/v3/users/06e3faddaeb74eb4b600bbfcad30e462` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:09.479316 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a097d4-b222-464e-ac27-9a236572d04d admin admin] RBAC: Authorizing `identity:delete_user(user_id=06e3faddaeb74eb4b600bbfcad30e462)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:09.482615 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a097d4-b222-464e-ac27-9a236572d04d admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:09.712435 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-a8a097d4-b222-464e-ac27-9a236572d04d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06e3faddaeb74eb4b600bbfcad30e462'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:09.721524 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-a8a097d4-b222-464e-ac27-9a236572d04d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06e3faddaeb74eb4b600bbfcad30e462'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:09.722256 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-a8a097d4-b222-464e-ac27-9a236572d04d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06e3faddaeb74eb4b600bbfcad30e462'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:09.743613 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-a8a097d4-b222-464e-ac27-9a236572d04d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06e3faddaeb74eb4b600bbfcad30e462'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:09.824454 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-a8a097d4-b222-464e-ac27-9a236572d04d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06e3faddaeb74eb4b600bbfcad30e462'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:09.831225 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1445/2841] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:52:09 2022] DELETE /identity/v3/users/06e3faddaeb74eb4b600bbfcad30e462 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:52:10.127672 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f168212-ae21-4dec-9a43-fad1238452f2 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:10.139638 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f168212-ae21-4dec-9a43-fad1238452f2 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:10.206333 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f206e95-b259-41fd-9b7a-ab5b7b4a9282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:10.207282 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f206e95-b259-41fd-9b7a-ab5b7b4a9282 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:10.207566 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f206e95-b259-41fd-9b7a-ab5b7b4a9282 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:10.207888 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f206e95-b259-41fd-9b7a-ab5b7b4a9282 admin admin] PATH_INFO: `/v3/projects/5d4c472a5e62449d9bc6a853be09f7eb` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:10.209077 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f206e95-b259-41fd-9b7a-ab5b7b4a9282 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d4c472a5e62449d9bc6a853be09f7eb)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:10.210963 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f206e95-b259-41fd-9b7a-ab5b7b4a9282 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:10.367139 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6f206e95-b259-41fd-9b7a-ab5b7b4a9282 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d4c472a5e62449d9bc6a853be09f7eb', 'request_id': 'req-6f206e95-b259-41fd-9b7a-ab5b7b4a9282'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:10.373572 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6f206e95-b259-41fd-9b7a-ab5b7b4a9282 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d4c472a5e62449d9bc6a853be09f7eb', 'request_id': 'req-6f206e95-b259-41fd-9b7a-ab5b7b4a9282'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:10.470737 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6f206e95-b259-41fd-9b7a-ab5b7b4a9282 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d4c472a5e62449d9bc6a853be09f7eb', 'request_id': 'req-6f206e95-b259-41fd-9b7a-ab5b7b4a9282'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:10.482309 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6f206e95-b259-41fd-9b7a-ab5b7b4a9282 admin admin] The token cache is being invalidate because project 5d4c472a5e62449d9bc6a853be09f7eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:52:10.482600 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6f206e95-b259-41fd-9b7a-ab5b7b4a9282 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:10.483363 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-6f206e95-b259-41fd-9b7a-ab5b7b4a9282 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:10.484958 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1397/2842] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:52:10 2022] DELETE /identity/v3/projects/5d4c472a5e62449d9bc6a853be09f7eb => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:52:10.490668 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e33428-a7f4-4bbf-ac3a-7e982e484c5b None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:10.491136 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e33428-a7f4-4bbf-ac3a-7e982e484c5b None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:10.491346 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e33428-a7f4-4bbf-ac3a-7e982e484c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:10.514757 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-97e33428-a7f4-4bbf-ac3a-7e982e484c5b None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:52:10.516302 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-97e33428-a7f4-4bbf-ac3a-7e982e484c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:10.544758 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:52:10.578387 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1446/2843] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:52:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:52:10.587185 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f01dac-6241-4b66-9ea9-01005fe528ed None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:10.587482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f01dac-6241-4b66-9ea9-01005fe528ed None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:10.589318 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f01dac-6241-4b66-9ea9-01005fe528ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:10.608606 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-23f01dac-6241-4b66-9ea9-01005fe528ed None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:52:10.609860 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-23f01dac-6241-4b66-9ea9-01005fe528ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:10.617950 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1398/2844] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:52:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:52:10.622327 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e33428-a7f4-4bbf-ac3a-7e982e484c5b None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:10.631855 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e33428-a7f4-4bbf-ac3a-7e982e484c5b None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:10.637256 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864a5c5-97af-433a-80a5-0610940dd1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:10.637927 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864a5c5-97af-433a-80a5-0610940dd1b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:10.638212 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864a5c5-97af-433a-80a5-0610940dd1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:10.638489 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864a5c5-97af-433a-80a5-0610940dd1b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:10.639222 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7864a5c5-97af-433a-80a5-0610940dd1b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:52:10.639570 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7864a5c5-97af-433a-80a5-0610940dd1b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:10.641165 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7864a5c5-97af-433a-80a5-0610940dd1b0 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:10.654567 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1447/2845] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:52:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:52:10.658958 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f01dac-6241-4b66-9ea9-01005fe528ed None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:10.664913 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:52:10.669076 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f01dac-6241-4b66-9ea9-01005fe528ed None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:10.673858 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e4f1e-dd07-4195-974a-d776277acd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:10.674514 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e4f1e-dd07-4195-974a-d776277acd33 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:10.674804 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e4f1e-dd07-4195-974a-d776277acd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:10.675198 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1e4f1e-dd07-4195-974a-d776277acd33 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:10.675921 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1e4f1e-dd07-4195-974a-d776277acd33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:10.677828 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1e4f1e-dd07-4195-974a-d776277acd33 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:10.719565 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864a5c5-97af-433a-80a5-0610940dd1b0 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:10.721455 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-7864a5c5-97af-433a-80a5-0610940dd1b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:10.788367 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1399/2846] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:52:10 2022] POST /identity/v3/projects => generated 391 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:52:10.789488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e4f1e-dd07-4195-974a-d776277acd33 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:10.793860 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864a5c5-97af-433a-80a5-0610940dd1b0 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:10.850515 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde91996-3f7a-4ce7-b7a7-cc749e98b3a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:10.851174 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde91996-3f7a-4ce7-b7a7-cc749e98b3a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:10.851510 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde91996-3f7a-4ce7-b7a7-cc749e98b3a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:10.852436 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde91996-3f7a-4ce7-b7a7-cc749e98b3a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:10.856231 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1e4f1e-dd07-4195-974a-d776277acd33 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:10.857005 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-cde91996-3f7a-4ce7-b7a7-cc749e98b3a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:10.861584 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5504e320-e35c-419f-a7b4-bd8c040387fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:10.862548 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504e320-e35c-419f-a7b4-bd8c040387fe admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:10.862841 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504e320-e35c-419f-a7b4-bd8c040387fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:10.863171 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504e320-e35c-419f-a7b4-bd8c040387fe admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:10.863892 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5504e320-e35c-419f-a7b4-bd8c040387fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:10.865935 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5504e320-e35c-419f-a7b4-bd8c040387fe admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:10.870430 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde91996-3f7a-4ce7-b7a7-cc749e98b3a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:10.871843 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde91996-3f7a-4ce7-b7a7-cc749e98b3a2 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:10.883534 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1448/2847] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:52:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:52:10.938830 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1400/2848] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:52:10 2022] POST /identity/v3/users => generated 296 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:52:10.942992 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde91996-3f7a-4ce7-b7a7-cc749e98b3a2 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:10.955486 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde91996-3f7a-4ce7-b7a7-cc749e98b3a2 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:10.960359 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e7a370-0fa9-471e-b0e9-64cf2f748eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:10.960958 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7a370-0fa9-471e-b0e9-64cf2f748eae admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:10.961248 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7a370-0fa9-471e-b0e9-64cf2f748eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:10.961524 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e7a370-0fa9-471e-b0e9-64cf2f748eae admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:10.962381 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e7a370-0fa9-471e-b0e9-64cf2f748eae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:10.964045 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e7a370-0fa9-471e-b0e9-64cf2f748eae admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:10.974715 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1449/2849] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:52:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:52:10.978789 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5504e320-e35c-419f-a7b4-bd8c040387fe admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:10.988830 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5504e320-e35c-419f-a7b4-bd8c040387fe admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:10.993521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349d0393-3316-479a-bb01-6d7d2a308a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:10.995657 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d0393-3316-479a-bb01-6d7d2a308a2c admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:10.995657 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d0393-3316-479a-bb01-6d7d2a308a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:10.995657 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d0393-3316-479a-bb01-6d7d2a308a2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:10.996619 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349d0393-3316-479a-bb01-6d7d2a308a2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:10.998233 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349d0393-3316-479a-bb01-6d7d2a308a2c admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:11.008450 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1401/2850] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:52:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:52:11.013253 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e7a370-0fa9-471e-b0e9-64cf2f748eae admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:11.023311 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e7a370-0fa9-471e-b0e9-64cf2f748eae admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:11.028776 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75e09be-c8e4-4c15-b5f4-6d1e654c1f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:11.029699 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e09be-c8e4-4c15-b5f4-6d1e654c1f13 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:11.029989 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e09be-c8e4-4c15-b5f4-6d1e654c1f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:11.030410 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75e09be-c8e4-4c15-b5f4-6d1e654c1f13 admin admin] PATH_INFO: `/v3/projects/95d8ece6637d48a98cdb1e07328f4997/users/c2347f0ba6bc46609971675903504606/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:11.045754 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75e09be-c8e4-4c15-b5f4-6d1e654c1f13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95d8ece6637d48a98cdb1e07328f4997, user_id=c2347f0ba6bc46609971675903504606, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:11.048465 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75e09be-c8e4-4c15-b5f4-6d1e654c1f13 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:11.116552 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1450/2851] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:52:11 2022] PUT /identity/v3/projects/95d8ece6637d48a98cdb1e07328f4997/users/c2347f0ba6bc46609971675903504606/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:52:11.126778 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ac415-b495-4b91-b665-758dcc734c19 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:11.127074 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ac415-b495-4b91-b665-758dcc734c19 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:11.127354 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ac415-b495-4b91-b665-758dcc734c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:11.156478 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-b61ac415-b495-4b91-b665-758dcc734c19 None None] MFA Rules not processed for user `c2347f0ba6bc46609971675903504606`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:52:11.157802 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-b61ac415-b495-4b91-b665-758dcc734c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:11.229641 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1402/2852] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:52:11 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:52:11.240803 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75e09be-c8e4-4c15-b5f4-6d1e654c1f13 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:11.242530 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-b75e09be-c8e4-4c15-b5f4-6d1e654c1f13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:11.306414 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75e09be-c8e4-4c15-b5f4-6d1e654c1f13 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:11.310390 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f3361a-68cb-43cf-b873-7798ddfdf973 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:11.311090 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3361a-68cb-43cf-b873-7798ddfdf973 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:11.311372 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3361a-68cb-43cf-b873-7798ddfdf973 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:11.311699 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f3361a-68cb-43cf-b873-7798ddfdf973 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:11.320668 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f3361a-68cb-43cf-b873-7798ddfdf973 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:11.322091 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f3361a-68cb-43cf-b873-7798ddfdf973 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:11.331534 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1451/2853] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:52:11 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:52:11.504564 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:52:11.781624 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61ac415-b495-4b91-b665-758dcc734c19 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:11.783905 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-b61ac415-b495-4b91-b665-758dcc734c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:11.876863 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61ac415-b495-4b91-b665-758dcc734c19 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:11.883123 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b42dd7b-3b64-4f45-a7dc-311566a01166 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:11.883864 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42dd7b-3b64-4f45-a7dc-311566a01166 service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:11.884145 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42dd7b-3b64-4f45-a7dc-311566a01166 service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:11.889620 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42dd7b-3b64-4f45-a7dc-311566a01166 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:11.889620 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-1b42dd7b-3b64-4f45-a7dc-311566a01166 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:11.899081 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b42dd7b-3b64-4f45-a7dc-311566a01166 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:11.900742 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b42dd7b-3b64-4f45-a7dc-311566a01166 service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:11.912488 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1403/2854] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:52:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 17:52:11.970724 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:52:19.549620 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f3361a-68cb-43cf-b873-7798ddfdf973 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:19.563375 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f3361a-68cb-43cf-b873-7798ddfdf973 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:19.569182 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee5a62-c343-465f-9c7b-2028992a1835 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:19.569924 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee5a62-c343-465f-9c7b-2028992a1835 service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:19.570278 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee5a62-c343-465f-9c7b-2028992a1835 service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:19.570660 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ee5a62-c343-465f-9c7b-2028992a1835 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:19.572631 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f6ee5a62-c343-465f-9c7b-2028992a1835 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:19.585527 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ee5a62-c343-465f-9c7b-2028992a1835 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:19.587129 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ee5a62-c343-465f-9c7b-2028992a1835 service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:19.599244 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1452/2855] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:52:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:52:22.817724 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=11 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:52:23.285978 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b42dd7b-3b64-4f45-a7dc-311566a01166 service nova] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:23.287375 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-1b42dd7b-3b64-4f45-a7dc-311566a01166 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:23.343390 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b42dd7b-3b64-4f45-a7dc-311566a01166 service nova] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:23.346977 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b90c78-ece2-464f-9f41-8b37661c9c33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:23.347651 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b90c78-ece2-464f-9f41-8b37661c9c33 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:23.347970 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b90c78-ece2-464f-9f41-8b37661c9c33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:23.348253 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b90c78-ece2-464f-9f41-8b37661c9c33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:23.350027 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-11b90c78-ece2-464f-9f41-8b37661c9c33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:23.410550 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b90c78-ece2-464f-9f41-8b37661c9c33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:23.412018 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b90c78-ece2-464f-9f41-8b37661c9c33 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:23.421739 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1404/2856] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:52:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:52:24.790681 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee5a62-c343-465f-9c7b-2028992a1835 service nova] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:24.792180 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f6ee5a62-c343-465f-9c7b-2028992a1835 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:24.813853 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ee5a62-c343-465f-9c7b-2028992a1835 service nova] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:24.813853 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5220b845-5246-4c8b-9a0e-59d5dd64f9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:24.813853 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220b845-5246-4c8b-9a0e-59d5dd64f9a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:24.813853 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220b845-5246-4c8b-9a0e-59d5dd64f9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:24.814209 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5220b845-5246-4c8b-9a0e-59d5dd64f9a8 admin admin] PATH_INFO: `/v3/users/6524c022e8ff47c69753868b15bc63d3` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:24.814209 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5220b845-5246-4c8b-9a0e-59d5dd64f9a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6524c022e8ff47c69753868b15bc63d3)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:24.815719 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5220b845-5246-4c8b-9a0e-59d5dd64f9a8 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:24.973078 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-5220b845-5246-4c8b-9a0e-59d5dd64f9a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6524c022e8ff47c69753868b15bc63d3'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:24.979834 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-5220b845-5246-4c8b-9a0e-59d5dd64f9a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6524c022e8ff47c69753868b15bc63d3'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:24.980476 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-5220b845-5246-4c8b-9a0e-59d5dd64f9a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6524c022e8ff47c69753868b15bc63d3'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:24.992174 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-5220b845-5246-4c8b-9a0e-59d5dd64f9a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6524c022e8ff47c69753868b15bc63d3'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:25.057283 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-5220b845-5246-4c8b-9a0e-59d5dd64f9a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6524c022e8ff47c69753868b15bc63d3'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:25.063798 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1453/2857] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:52:24 2022] DELETE /identity/v3/users/6524c022e8ff47c69753868b15bc63d3 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:52:25.333614 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b90c78-ece2-464f-9f41-8b37661c9c33 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:25.343837 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b90c78-ece2-464f-9f41-8b37661c9c33 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:25.401120 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fde5a91-615f-4981-9e51-c67cbb17362f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:25.401976 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde5a91-615f-4981-9e51-c67cbb17362f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:25.402259 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde5a91-615f-4981-9e51-c67cbb17362f admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:25.402533 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fde5a91-615f-4981-9e51-c67cbb17362f admin admin] PATH_INFO: `/v3/projects/ca94c8f670004e8b8d70d61b5aeee7c8` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:25.403786 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fde5a91-615f-4981-9e51-c67cbb17362f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca94c8f670004e8b8d70d61b5aeee7c8)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:25.405670 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fde5a91-615f-4981-9e51-c67cbb17362f admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:25.624028 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-7fde5a91-615f-4981-9e51-c67cbb17362f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca94c8f670004e8b8d70d61b5aeee7c8', 'request_id': 'req-7fde5a91-615f-4981-9e51-c67cbb17362f'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:25.625793 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:52:25.630565 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-7fde5a91-615f-4981-9e51-c67cbb17362f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca94c8f670004e8b8d70d61b5aeee7c8', 'request_id': 'req-7fde5a91-615f-4981-9e51-c67cbb17362f'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:25.709236 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-7fde5a91-615f-4981-9e51-c67cbb17362f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca94c8f670004e8b8d70d61b5aeee7c8', 'request_id': 'req-7fde5a91-615f-4981-9e51-c67cbb17362f'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:25.726361 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-7fde5a91-615f-4981-9e51-c67cbb17362f admin admin] The token cache is being invalidate because project ca94c8f670004e8b8d70d61b5aeee7c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:52:25.726656 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-7fde5a91-615f-4981-9e51-c67cbb17362f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:25.727309 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-7fde5a91-615f-4981-9e51-c67cbb17362f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:25.728927 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1405/2858] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:52:25 2022] DELETE /identity/v3/projects/ca94c8f670004e8b8d70d61b5aeee7c8 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:52:25.732828 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5220b845-5246-4c8b-9a0e-59d5dd64f9a8 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:25.734450 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-5220b845-5246-4c8b-9a0e-59d5dd64f9a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:25.748264 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5220b845-5246-4c8b-9a0e-59d5dd64f9a8 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:25.804658 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657bca6a-2abd-4384-8dd0-f4f9fe7d4dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:25.805530 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657bca6a-2abd-4384-8dd0-f4f9fe7d4dcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:25.805813 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657bca6a-2abd-4384-8dd0-f4f9fe7d4dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:25.806090 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657bca6a-2abd-4384-8dd0-f4f9fe7d4dcc admin admin] PATH_INFO: `/v3/users/ababfd0f0c0a440192b3727c005ef6b6` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:25.807270 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657bca6a-2abd-4384-8dd0-f4f9fe7d4dcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ababfd0f0c0a440192b3727c005ef6b6)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:25.809227 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657bca6a-2abd-4384-8dd0-f4f9fe7d4dcc admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:26.007756 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:52:26.009556 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-657bca6a-2abd-4384-8dd0-f4f9fe7d4dcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ababfd0f0c0a440192b3727c005ef6b6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:26.017059 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-657bca6a-2abd-4384-8dd0-f4f9fe7d4dcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ababfd0f0c0a440192b3727c005ef6b6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:26.017722 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-657bca6a-2abd-4384-8dd0-f4f9fe7d4dcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ababfd0f0c0a440192b3727c005ef6b6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:26.029319 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-657bca6a-2abd-4384-8dd0-f4f9fe7d4dcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ababfd0f0c0a440192b3727c005ef6b6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:26.114899 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-657bca6a-2abd-4384-8dd0-f4f9fe7d4dcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ababfd0f0c0a440192b3727c005ef6b6'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:26.121220 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1454/2859] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:52:25 2022] DELETE /identity/v3/users/ababfd0f0c0a440192b3727c005ef6b6 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:52:26.401877 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fde5a91-615f-4981-9e51-c67cbb17362f admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:26.403424 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-7fde5a91-615f-4981-9e51-c67cbb17362f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:26.421885 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fde5a91-615f-4981-9e51-c67cbb17362f admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:26.483515 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cb5650-b0d6-43ce-8064-c900d3275dab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:26.484418 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb5650-b0d6-43ce-8064-c900d3275dab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:26.484821 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb5650-b0d6-43ce-8064-c900d3275dab admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:26.484987 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb5650-b0d6-43ce-8064-c900d3275dab admin admin] PATH_INFO: `/v3/projects/9e8eb07045034c35bfc091cd46d0c152` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:26.486190 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cb5650-b0d6-43ce-8064-c900d3275dab admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e8eb07045034c35bfc091cd46d0c152)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:26.488148 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cb5650-b0d6-43ce-8064-c900d3275dab admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:26.686593 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-18cb5650-b0d6-43ce-8064-c900d3275dab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e8eb07045034c35bfc091cd46d0c152', 'request_id': 'req-18cb5650-b0d6-43ce-8064-c900d3275dab'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:26.692716 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-18cb5650-b0d6-43ce-8064-c900d3275dab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e8eb07045034c35bfc091cd46d0c152', 'request_id': 'req-18cb5650-b0d6-43ce-8064-c900d3275dab'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:26.852305 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-18cb5650-b0d6-43ce-8064-c900d3275dab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e8eb07045034c35bfc091cd46d0c152', 'request_id': 'req-18cb5650-b0d6-43ce-8064-c900d3275dab'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:26.865948 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-18cb5650-b0d6-43ce-8064-c900d3275dab admin admin] The token cache is being invalidate because project 9e8eb07045034c35bfc091cd46d0c152 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:52:26.866243 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-18cb5650-b0d6-43ce-8064-c900d3275dab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:26.867000 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-18cb5650-b0d6-43ce-8064-c900d3275dab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:26.868943 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1406/2860] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:52:26 2022] DELETE /identity/v3/projects/9e8eb07045034c35bfc091cd46d0c152 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:52:26.873936 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910d632-788b-4da3-af63-04cd15e7ca0e None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:26.874909 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910d632-788b-4da3-af63-04cd15e7ca0e None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:26.875478 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910d632-788b-4da3-af63-04cd15e7ca0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:26.897267 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-d910d632-788b-4da3-af63-04cd15e7ca0e None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:52:26.898488 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d910d632-788b-4da3-af63-04cd15e7ca0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:26.959051 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1455/2861] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:52:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:52:26.966763 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f803a45a-6635-4e95-9f87-5f84e40a00cc None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:26.967028 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f803a45a-6635-4e95-9f87-5f84e40a00cc None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:26.967313 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f803a45a-6635-4e95-9f87-5f84e40a00cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:26.988747 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-f803a45a-6635-4e95-9f87-5f84e40a00cc None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:52:26.989959 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-f803a45a-6635-4e95-9f87-5f84e40a00cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:26.997890 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1407/2862] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:52:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:52:27.003091 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d910d632-788b-4da3-af63-04cd15e7ca0e None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:27.013058 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d910d632-788b-4da3-af63-04cd15e7ca0e None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:27.017777 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b725ae9-7687-4fd5-8035-1084b0d7b7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:27.018434 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b725ae9-7687-4fd5-8035-1084b0d7b7e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:27.018716 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b725ae9-7687-4fd5-8035-1084b0d7b7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:27.018991 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b725ae9-7687-4fd5-8035-1084b0d7b7e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:27.019714 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b725ae9-7687-4fd5-8035-1084b0d7b7e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80469) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:52:27.020049 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b725ae9-7687-4fd5-8035-1084b0d7b7e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:27.021620 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b725ae9-7687-4fd5-8035-1084b0d7b7e2 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:27.034070 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1456/2863] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:52:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:52:27.038444 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f803a45a-6635-4e95-9f87-5f84e40a00cc None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:27.049780 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f803a45a-6635-4e95-9f87-5f84e40a00cc None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:27.054512 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d1ec7c-d472-42cf-9e8c-7e79cc72a5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:27.055237 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d1ec7c-d472-42cf-9e8c-7e79cc72a5a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:27.055591 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d1ec7c-d472-42cf-9e8c-7e79cc72a5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:27.055981 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d1ec7c-d472-42cf-9e8c-7e79cc72a5a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:27.056747 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d1ec7c-d472-42cf-9e8c-7e79cc72a5a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:27.058665 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d1ec7c-d472-42cf-9e8c-7e79cc72a5a3 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:27.171238 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1408/2864] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:52:27 2022] POST /identity/v3/projects => generated 373 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:52:27.173024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b725ae9-7687-4fd5-8035-1084b0d7b7e2 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:27.183130 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b725ae9-7687-4fd5-8035-1084b0d7b7e2 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:27.243927 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad87d7-c86f-46ed-bad6-b57522f6a6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:27.244612 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad87d7-c86f-46ed-bad6-b57522f6a6d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:27.244899 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad87d7-c86f-46ed-bad6-b57522f6a6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:27.245180 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad87d7-c86f-46ed-bad6-b57522f6a6d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:27.245889 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ad87d7-c86f-46ed-bad6-b57522f6a6d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:27.247806 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ad87d7-c86f-46ed-bad6-b57522f6a6d7 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:27.418589 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1457/2865] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:52:27 2022] POST /identity/v3/users => generated 287 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:52:27.422535 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d1ec7c-d472-42cf-9e8c-7e79cc72a5a3 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:27.433242 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d1ec7c-d472-42cf-9e8c-7e79cc72a5a3 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:27.440317 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcdf0c5-b936-42e4-b965-65b8baa7c90f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:27.440933 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcdf0c5-b936-42e4-b965-65b8baa7c90f admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:27.441222 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcdf0c5-b936-42e4-b965-65b8baa7c90f admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:27.441501 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcdf0c5-b936-42e4-b965-65b8baa7c90f admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:27.442275 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcdf0c5-b936-42e4-b965-65b8baa7c90f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:27.443895 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcdf0c5-b936-42e4-b965-65b8baa7c90f admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:27.455227 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1409/2866] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:52:27.459266 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad87d7-c86f-46ed-bad6-b57522f6a6d7 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:27.469931 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad87d7-c86f-46ed-bad6-b57522f6a6d7 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:27.474638 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f88567-40ac-4a32-b461-222d8be489f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:27.475298 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f88567-40ac-4a32-b461-222d8be489f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:27.475578 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f88567-40ac-4a32-b461-222d8be489f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:27.475890 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f88567-40ac-4a32-b461-222d8be489f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:27.476608 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f88567-40ac-4a32-b461-222d8be489f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:27.478168 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f88567-40ac-4a32-b461-222d8be489f7 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:27.488060 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1458/2867] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:52:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:52:27.492001 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcdf0c5-b936-42e4-b965-65b8baa7c90f admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:27.501945 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcdf0c5-b936-42e4-b965-65b8baa7c90f admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:27.506519 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bad6ca-3dc5-4761-bd28-9cdaecd92d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:27.507482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bad6ca-3dc5-4761-bd28-9cdaecd92d34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:27.507783 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bad6ca-3dc5-4761-bd28-9cdaecd92d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:27.508226 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bad6ca-3dc5-4761-bd28-9cdaecd92d34 admin admin] PATH_INFO: `/v3/projects/66c963fbbfd44e70ac6e5cca8d7ca292/users/782062228302471f83ba6860b7162f47/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:27.523190 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bad6ca-3dc5-4761-bd28-9cdaecd92d34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66c963fbbfd44e70ac6e5cca8d7ca292, user_id=782062228302471f83ba6860b7162f47, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:27.525960 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bad6ca-3dc5-4761-bd28-9cdaecd92d34 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:27.625292 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1410/2868] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:52:27 2022] PUT /identity/v3/projects/66c963fbbfd44e70ac6e5cca8d7ca292/users/782062228302471f83ba6860b7162f47/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:52:27.637224 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76227d-6c78-4ac6-8c7e-69a7d5c80d6f None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:27.637568 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76227d-6c78-4ac6-8c7e-69a7d5c80d6f None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:27.637864 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc76227d-6c78-4ac6-8c7e-69a7d5c80d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:27.664592 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-dc76227d-6c78-4ac6-8c7e-69a7d5c80d6f None None] MFA Rules not processed for user `782062228302471f83ba6860b7162f47`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:52:27.665796 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-dc76227d-6c78-4ac6-8c7e-69a7d5c80d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:27.682535 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:52:27.731162 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1459/2869] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:52:27 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:52:27.742533 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bad6ca-3dc5-4761-bd28-9cdaecd92d34 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:27.744324 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-08bad6ca-3dc5-4761-bd28-9cdaecd92d34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:27.754592 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:52:27.801435 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bad6ca-3dc5-4761-bd28-9cdaecd92d34 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:27.805123 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f34d1c4-ab63-477d-8939-772adeb4bda6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:27.809477 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34d1c4-ab63-477d-8939-772adeb4bda6 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:27.809477 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34d1c4-ab63-477d-8939-772adeb4bda6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:27.809477 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34d1c4-ab63-477d-8939-772adeb4bda6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:27.815139 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f34d1c4-ab63-477d-8939-772adeb4bda6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:27.816615 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f34d1c4-ab63-477d-8939-772adeb4bda6 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:27.825264 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1411/2870] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:52:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:52:30.217878 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d37d07-5399-4007-9bdc-cb09780069d6 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:30.218377 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d37d07-5399-4007-9bdc-cb09780069d6 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:30.218468 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d37d07-5399-4007-9bdc-cb09780069d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:30.236081 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-45d37d07-5399-4007-9bdc-cb09780069d6 None None] MFA Rules not processed for user `f437559c9a344df7bde785d5e494516b`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:52:30.238066 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-45d37d07-5399-4007-9bdc-cb09780069d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:30.297100 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1460/2871] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:52:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:52:31.984031 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f34d1c4-ab63-477d-8939-772adeb4bda6 service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:32.046191 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f34d1c4-ab63-477d-8939-772adeb4bda6 service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:32.051355 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b051b93d-3c09-41fd-93bb-82fa7e202e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:32.052259 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051b93d-3c09-41fd-93bb-82fa7e202e12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:32.052546 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051b93d-3c09-41fd-93bb-82fa7e202e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:32.052824 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b051b93d-3c09-41fd-93bb-82fa7e202e12 admin admin] PATH_INFO: `/v3/users/782062228302471f83ba6860b7162f47` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:32.054038 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b051b93d-3c09-41fd-93bb-82fa7e202e12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=782062228302471f83ba6860b7162f47)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:32.055980 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b051b93d-3c09-41fd-93bb-82fa7e202e12 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:32.271780 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d37d07-5399-4007-9bdc-cb09780069d6 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:32.272784 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-b051b93d-3c09-41fd-93bb-82fa7e202e12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '782062228302471f83ba6860b7162f47'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:32.273347 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-45d37d07-5399-4007-9bdc-cb09780069d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:32.285916 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-b051b93d-3c09-41fd-93bb-82fa7e202e12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '782062228302471f83ba6860b7162f47'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:32.287607 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d37d07-5399-4007-9bdc-cb09780069d6 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:32.292237 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380b52b-1cb8-4efe-b606-c7323bc9a142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:32.293443 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380b52b-1cb8-4efe-b606-c7323bc9a142 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:32.293554 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-b051b93d-3c09-41fd-93bb-82fa7e202e12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '782062228302471f83ba6860b7162f47'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:32.293897 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380b52b-1cb8-4efe-b606-c7323bc9a142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:32.294174 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380b52b-1cb8-4efe-b606-c7323bc9a142 admin admin] PATH_INFO: `/v3/users/c2347f0ba6bc46609971675903504606` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:32.294276 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-b051b93d-3c09-41fd-93bb-82fa7e202e12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '782062228302471f83ba6860b7162f47'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:32.295489 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9380b52b-1cb8-4efe-b606-c7323bc9a142 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2347f0ba6bc46609971675903504606)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:32.298070 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9380b52b-1cb8-4efe-b606-c7323bc9a142 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:32.400718 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-b051b93d-3c09-41fd-93bb-82fa7e202e12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '782062228302471f83ba6860b7162f47'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:32.408714 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1412/2872] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:52:31 2022] DELETE /identity/v3/users/782062228302471f83ba6860b7162f47 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:52:32.413042 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70f521-0dda-45cf-bd24-ede9a925d229 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:32.413335 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70f521-0dda-45cf-bd24-ede9a925d229 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:32.413614 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c70f521-0dda-45cf-bd24-ede9a925d229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:32.431432 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-2c70f521-0dda-45cf-bd24-ede9a925d229 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:52:32.432642 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-2c70f521-0dda-45cf-bd24-ede9a925d229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:32.455767 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:52:32.491435 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1413/2873] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:52:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:52:32.501312 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c70f521-0dda-45cf-bd24-ede9a925d229 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:32.502649 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-2c70f521-0dda-45cf-bd24-ede9a925d229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:32.548431 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-9380b52b-1cb8-4efe-b606-c7323bc9a142 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2347f0ba6bc46609971675903504606'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:32.556260 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-9380b52b-1cb8-4efe-b606-c7323bc9a142 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2347f0ba6bc46609971675903504606'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:32.557079 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-9380b52b-1cb8-4efe-b606-c7323bc9a142 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2347f0ba6bc46609971675903504606'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:32.564081 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c70f521-0dda-45cf-bd24-ede9a925d229 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:32.568128 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3589b9ad-ab14-41a2-8734-aa9b58b9f735 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:32.568787 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589b9ad-ab14-41a2-8734-aa9b58b9f735 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:32.569081 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589b9ad-ab14-41a2-8734-aa9b58b9f735 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:32.569340 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589b9ad-ab14-41a2-8734-aa9b58b9f735 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:32.570034 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-9380b52b-1cb8-4efe-b606-c7323bc9a142 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2347f0ba6bc46609971675903504606'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:32.579380 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3589b9ad-ab14-41a2-8734-aa9b58b9f735 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:32.581039 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3589b9ad-ab14-41a2-8734-aa9b58b9f735 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:32.590456 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1414/2874] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:52:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 17:52:32.680483 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-9380b52b-1cb8-4efe-b606-c7323bc9a142 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2347f0ba6bc46609971675903504606'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:32.687147 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1461/2875] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:52:32 2022] DELETE /identity/v3/users/c2347f0ba6bc46609971675903504606 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:52:32.695570 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3dc886-4fb1-4ac1-b3a6-dfe89683d162 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:32.695570 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3dc886-4fb1-4ac1-b3a6-dfe89683d162 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:32.695570 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3dc886-4fb1-4ac1-b3a6-dfe89683d162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:32.709687 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-2b3dc886-4fb1-4ac1-b3a6-dfe89683d162 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:52:32.710878 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-2b3dc886-4fb1-4ac1-b3a6-dfe89683d162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:32.769120 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1415/2876] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:52:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:52:32.778706 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380b52b-1cb8-4efe-b606-c7323bc9a142 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:32.780179 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-9380b52b-1cb8-4efe-b606-c7323bc9a142 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:32.824966 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:52:32.864655 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380b52b-1cb8-4efe-b606-c7323bc9a142 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:32.868772 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c41d21e-04ec-479d-a0e8-7c6c36835712 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:32.869430 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c41d21e-04ec-479d-a0e8-7c6c36835712 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:32.869717 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c41d21e-04ec-479d-a0e8-7c6c36835712 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:32.869998 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c41d21e-04ec-479d-a0e8-7c6c36835712 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:32.886773 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c41d21e-04ec-479d-a0e8-7c6c36835712 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:32.888469 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c41d21e-04ec-479d-a0e8-7c6c36835712 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:32.899187 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1462/2877] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:52:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:52:33.316889 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3dc886-4fb1-4ac1-b3a6-dfe89683d162 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:33.318934 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-2b3dc886-4fb1-4ac1-b3a6-dfe89683d162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:33.333495 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3dc886-4fb1-4ac1-b3a6-dfe89683d162 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:33.339943 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1132a6-ab7d-443a-b779-733686966a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:33.341023 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1132a6-ab7d-443a-b779-733686966a84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:33.341381 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1132a6-ab7d-443a-b779-733686966a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:33.342020 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1132a6-ab7d-443a-b779-733686966a84 admin admin] PATH_INFO: `/v3/projects/66c963fbbfd44e70ac6e5cca8d7ca292` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:33.343814 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1132a6-ab7d-443a-b779-733686966a84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66c963fbbfd44e70ac6e5cca8d7ca292)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:33.345870 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1132a6-ab7d-443a-b779-733686966a84 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:33.453925 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c41d21e-04ec-479d-a0e8-7c6c36835712 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:33.455548 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-4c41d21e-04ec-479d-a0e8-7c6c36835712 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:33.476150 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c41d21e-04ec-479d-a0e8-7c6c36835712 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:33.491465 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-1b1132a6-ab7d-443a-b779-733686966a84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66c963fbbfd44e70ac6e5cca8d7ca292', 'request_id': 'req-1b1132a6-ab7d-443a-b779-733686966a84'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:33.497883 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-1b1132a6-ab7d-443a-b779-733686966a84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66c963fbbfd44e70ac6e5cca8d7ca292', 'request_id': 'req-1b1132a6-ab7d-443a-b779-733686966a84'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:33.542134 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7032b80-8474-4d78-8b65-7f13afba8c13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:33.543201 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7032b80-8474-4d78-8b65-7f13afba8c13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:33.543487 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7032b80-8474-4d78-8b65-7f13afba8c13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:33.543815 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7032b80-8474-4d78-8b65-7f13afba8c13 admin admin] PATH_INFO: `/v3/projects/95d8ece6637d48a98cdb1e07328f4997` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:33.545022 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7032b80-8474-4d78-8b65-7f13afba8c13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95d8ece6637d48a98cdb1e07328f4997)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:33.546925 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7032b80-8474-4d78-8b65-7f13afba8c13 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:33.671565 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-1b1132a6-ab7d-443a-b779-733686966a84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66c963fbbfd44e70ac6e5cca8d7ca292', 'request_id': 'req-1b1132a6-ab7d-443a-b779-733686966a84'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:33.677711 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:52:33.691753 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-1b1132a6-ab7d-443a-b779-733686966a84 admin admin] The token cache is being invalidate because project 66c963fbbfd44e70ac6e5cca8d7ca292 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:52:33.691753 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-1b1132a6-ab7d-443a-b779-733686966a84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:33.691753 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-1b1132a6-ab7d-443a-b779-733686966a84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:33.691753 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1416/2878] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:52:33 2022] DELETE /identity/v3/projects/66c963fbbfd44e70ac6e5cca8d7ca292 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:52:33.692578 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-c7032b80-8474-4d78-8b65-7f13afba8c13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95d8ece6637d48a98cdb1e07328f4997', 'request_id': 'req-c7032b80-8474-4d78-8b65-7f13afba8c13'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:33.694047 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07a6b5-c479-4148-bb78-e0d3ca7b01ce None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:33.694411 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07a6b5-c479-4148-bb78-e0d3ca7b01ce None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:33.695327 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07a6b5-c479-4148-bb78-e0d3ca7b01ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:33.699524 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-c7032b80-8474-4d78-8b65-7f13afba8c13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95d8ece6637d48a98cdb1e07328f4997', 'request_id': 'req-c7032b80-8474-4d78-8b65-7f13afba8c13'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:33.707889 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-c7032b80-8474-4d78-8b65-7f13afba8c13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95d8ece6637d48a98cdb1e07328f4997', 'request_id': 'req-c7032b80-8474-4d78-8b65-7f13afba8c13'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:33.715003 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-bb07a6b5-c479-4148-bb78-e0d3ca7b01ce None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:52:33.720690 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-bb07a6b5-c479-4148-bb78-e0d3ca7b01ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:33.772508 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-c7032b80-8474-4d78-8b65-7f13afba8c13 admin admin] The token cache is being invalidate because project 95d8ece6637d48a98cdb1e07328f4997 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:52:33.772895 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-c7032b80-8474-4d78-8b65-7f13afba8c13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:33.773598 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-c7032b80-8474-4d78-8b65-7f13afba8c13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:52:33.775458 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1463/2879] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:52:33 2022] DELETE /identity/v3/projects/95d8ece6637d48a98cdb1e07328f4997 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:52:33.780480 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9c2ad-9228-4b23-bd06-77ff80263599 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:33.781245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9c2ad-9228-4b23-bd06-77ff80263599 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:33.781245 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9c2ad-9228-4b23-bd06-77ff80263599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:33.783428 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1417/2880] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:52:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:52:33.791906 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5140cc0-a080-449d-bc95-f85abf0ef90d None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:33.792233 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5140cc0-a080-449d-bc95-f85abf0ef90d None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:33.792517 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5140cc0-a080-449d-bc95-f85abf0ef90d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:33.807514 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-d5a9c2ad-9228-4b23-bd06-77ff80263599 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:52:33.809024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d5a9c2ad-9228-4b23-bd06-77ff80263599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:33.811982 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-c5140cc0-a080-449d-bc95-f85abf0ef90d None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:52:33.813244 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-c5140cc0-a080-449d-bc95-f85abf0ef90d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:33.818560 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1464/2881] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:52:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:52:33.827932 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba4b78-a611-4450-bc79-3ba8c4e3fe25 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:33.828085 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1418/2882] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:52:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:52:33.828920 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba4b78-a611-4450-bc79-3ba8c4e3fe25 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:33.829281 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cba4b78-a611-4450-bc79-3ba8c4e3fe25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:33.832174 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5140cc0-a080-449d-bc95-f85abf0ef90d None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:33.841521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5140cc0-a080-449d-bc95-f85abf0ef90d None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:33.847431 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08635889-1bd9-40f0-a83c-b5a8cb3b9894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:33.848134 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08635889-1bd9-40f0-a83c-b5a8cb3b9894 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:33.848432 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08635889-1bd9-40f0-a83c-b5a8cb3b9894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:33.848706 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08635889-1bd9-40f0-a83c-b5a8cb3b9894 admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:33.849442 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08635889-1bd9-40f0-a83c-b5a8cb3b9894 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:52:33.849752 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08635889-1bd9-40f0-a83c-b5a8cb3b9894 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:33.851353 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08635889-1bd9-40f0-a83c-b5a8cb3b9894 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:33.853302 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-5cba4b78-a611-4450-bc79-3ba8c4e3fe25 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:52:33.857654 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-5cba4b78-a611-4450-bc79-3ba8c4e3fe25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:33.864066 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1419/2883] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:52:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 06 17:52:33.865394 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1465/2884] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:52:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:52:33.868582 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cba4b78-a611-4450-bc79-3ba8c4e3fe25 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:33.871029 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08635889-1bd9-40f0-a83c-b5a8cb3b9894 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:33.878846 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cba4b78-a611-4450-bc79-3ba8c4e3fe25 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:33.884047 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94047ca-6792-40b3-b58a-b48e1fcde085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:33.886349 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94047ca-6792-40b3-b58a-b48e1fcde085 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:33.886349 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94047ca-6792-40b3-b58a-b48e1fcde085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:33.886349 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94047ca-6792-40b3-b58a-b48e1fcde085 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:33.886628 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08635889-1bd9-40f0-a83c-b5a8cb3b9894 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:33.886726 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94047ca-6792-40b3-b58a-b48e1fcde085 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:33.887896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94047ca-6792-40b3-b58a-b48e1fcde085 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:33.891340 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd940f99-6b1e-4ba5-a258-3df1b85e3b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:33.892995 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd940f99-6b1e-4ba5-a258-3df1b85e3b0f admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:33.892995 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd940f99-6b1e-4ba5-a258-3df1b85e3b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:33.892995 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd940f99-6b1e-4ba5-a258-3df1b85e3b0f admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:33.893406 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd940f99-6b1e-4ba5-a258-3df1b85e3b0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:52:33.893726 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd940f99-6b1e-4ba5-a258-3df1b85e3b0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:33.895334 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd940f99-6b1e-4ba5-a258-3df1b85e3b0f admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:33.908603 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1420/2885] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:52:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:52:33.912835 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd940f99-6b1e-4ba5-a258-3df1b85e3b0f admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:33.923453 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd940f99-6b1e-4ba5-a258-3df1b85e3b0f admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:33.928172 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a43c2c-3881-447c-8649-0bfbfbcec874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:33.928820 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a43c2c-3881-447c-8649-0bfbfbcec874 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:33.929133 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a43c2c-3881-447c-8649-0bfbfbcec874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:33.929409 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a43c2c-3881-447c-8649-0bfbfbcec874 admin admin] PATH_INFO: `/v3/projects` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:33.930119 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a43c2c-3881-447c-8649-0bfbfbcec874 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:33.932041 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a43c2c-3881-447c-8649-0bfbfbcec874 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:33.979406 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1466/2886] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:52:33 2022] POST /identity/v3/projects => generated 361 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:52:33.983576 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94047ca-6792-40b3-b58a-b48e1fcde085 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:34.017438 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:52:34.045776 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94047ca-6792-40b3-b58a-b48e1fcde085 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:34.050972 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d7507-fd98-46b5-8247-61a2b5ef1ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:34.051643 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d7507-fd98-46b5-8247-61a2b5ef1ec0 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:34.051953 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d7507-fd98-46b5-8247-61a2b5ef1ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:34.052223 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d7507-fd98-46b5-8247-61a2b5ef1ec0 admin admin] PATH_INFO: `/v3/users` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:34.052912 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678d7507-fd98-46b5-8247-61a2b5ef1ec0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:34.056968 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678d7507-fd98-46b5-8247-61a2b5ef1ec0 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:34.060970 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1421/2887] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:52:33 2022] POST /identity/v3/projects => generated 379 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:52:34.064874 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a43c2c-3881-447c-8649-0bfbfbcec874 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:34.131954 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a43c2c-3881-447c-8649-0bfbfbcec874 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:34.136991 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a73c3-45b5-42da-9e7c-333d71e1d819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:34.137679 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a73c3-45b5-42da-9e7c-333d71e1d819 admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:34.137968 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a73c3-45b5-42da-9e7c-333d71e1d819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:34.138250 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a73c3-45b5-42da-9e7c-333d71e1d819 admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:34.139193 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71a73c3-45b5-42da-9e7c-333d71e1d819 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:34.142142 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71a73c3-45b5-42da-9e7c-333d71e1d819 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:34.157252 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1467/2888] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:52:33 2022] POST /identity/v3/users => generated 281 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:52:34.161467 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d7507-fd98-46b5-8247-61a2b5ef1ec0 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:34.171691 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d7507-fd98-46b5-8247-61a2b5ef1ec0 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:34.176658 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7713d0e3-3898-46fb-8ad8-95dcadc0773f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:34.177314 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7713d0e3-3898-46fb-8ad8-95dcadc0773f admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:34.177596 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7713d0e3-3898-46fb-8ad8-95dcadc0773f admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:34.177878 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7713d0e3-3898-46fb-8ad8-95dcadc0773f admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:34.178599 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7713d0e3-3898-46fb-8ad8-95dcadc0773f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:34.180207 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7713d0e3-3898-46fb-8ad8-95dcadc0773f admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:34.190070 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1468/2889] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:52:34.193977 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7713d0e3-3898-46fb-8ad8-95dcadc0773f admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:34.203055 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7713d0e3-3898-46fb-8ad8-95dcadc0773f admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:34.208157 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc39f9-213b-49e4-948c-412460b8a093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:34.208756 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc39f9-213b-49e4-948c-412460b8a093 admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:34.209042 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc39f9-213b-49e4-948c-412460b8a093 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:34.209318 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc39f9-213b-49e4-948c-412460b8a093 admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:34.210034 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fc39f9-213b-49e4-948c-412460b8a093 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:34.211608 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fc39f9-213b-49e4-948c-412460b8a093 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:34.221708 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1469/2890] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:52:34.225458 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc39f9-213b-49e4-948c-412460b8a093 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:34.235881 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc39f9-213b-49e4-948c-412460b8a093 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:34.240497 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b1adde-8099-406c-99b8-5791b922cd7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:34.241359 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1adde-8099-406c-99b8-5791b922cd7c admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:34.241639 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1adde-8099-406c-99b8-5791b922cd7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:34.241906 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b1adde-8099-406c-99b8-5791b922cd7c admin admin] PATH_INFO: `/v3/projects/657f67d43f59480e82347efca1023898/users/c1c1057cc8e04cefacda126daf32babf/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:34.248233 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1422/2891] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:52:34 2022] POST /identity/v3/users => generated 290 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:52:34.254156 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a73c3-45b5-42da-9e7c-333d71e1d819 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:34.256711 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b1adde-8099-406c-99b8-5791b922cd7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=657f67d43f59480e82347efca1023898, user_id=c1c1057cc8e04cefacda126daf32babf, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:34.259348 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b1adde-8099-406c-99b8-5791b922cd7c admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:34.264145 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a73c3-45b5-42da-9e7c-333d71e1d819 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:34.269343 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a47fac-c589-4ac9-b163-909380a5e635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:34.274198 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a47fac-c589-4ac9-b163-909380a5e635 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:34.274198 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a47fac-c589-4ac9-b163-909380a5e635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:34.274198 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a47fac-c589-4ac9-b163-909380a5e635 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:34.274198 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a47fac-c589-4ac9-b163-909380a5e635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:34.274198 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a47fac-c589-4ac9-b163-909380a5e635 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:34.284228 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1423/2892] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:52:34.287887 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a47fac-c589-4ac9-b163-909380a5e635 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:34.299379 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a47fac-c589-4ac9-b163-909380a5e635 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:34.304927 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f94da-0055-4ef4-b57a-37a59fe68037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:34.306229 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f94da-0055-4ef4-b57a-37a59fe68037 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:34.306229 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f94da-0055-4ef4-b57a-37a59fe68037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:34.306685 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f94da-0055-4ef4-b57a-37a59fe68037 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:34.307677 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f94da-0055-4ef4-b57a-37a59fe68037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:34.309372 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2f94da-0055-4ef4-b57a-37a59fe68037 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:34.321325 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1424/2893] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:52:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:52:34.326052 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f94da-0055-4ef4-b57a-37a59fe68037 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:34.338087 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2f94da-0055-4ef4-b57a-37a59fe68037 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:34.345358 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea31517-4b9e-4019-ba57-c7ef784893ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:34.345917 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea31517-4b9e-4019-ba57-c7ef784893ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:34.346404 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea31517-4b9e-4019-ba57-c7ef784893ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:34.346753 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea31517-4b9e-4019-ba57-c7ef784893ec admin admin] PATH_INFO: `/v3/projects/3d04d8f5cfdd466191e5c9e6fc5fc25e/users/d2c4d7dd66614e52a006d1c6fb9d2fec/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:34.350501 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:52:34.363018 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1470/2894] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:52:34 2022] PUT /identity/v3/projects/657f67d43f59480e82347efca1023898/users/c1c1057cc8e04cefacda126daf32babf/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:52:34.364830 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea31517-4b9e-4019-ba57-c7ef784893ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d04d8f5cfdd466191e5c9e6fc5fc25e, user_id=d2c4d7dd66614e52a006d1c6fb9d2fec, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:34.367679 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea31517-4b9e-4019-ba57-c7ef784893ec admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:34.373355 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a24d8f-45f5-49a9-ac64-d1a182f5449b None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:34.373919 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a24d8f-45f5-49a9-ac64-d1a182f5449b None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:34.374250 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a24d8f-45f5-49a9-ac64-d1a182f5449b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:34.404560 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-62a24d8f-45f5-49a9-ac64-d1a182f5449b None None] MFA Rules not processed for user `c1c1057cc8e04cefacda126daf32babf`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:52:34.405797 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-62a24d8f-45f5-49a9-ac64-d1a182f5449b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:34.470539 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1425/2895] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:52:34 2022] PUT /identity/v3/projects/3d04d8f5cfdd466191e5c9e6fc5fc25e/users/d2c4d7dd66614e52a006d1c6fb9d2fec/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:52:34.476807 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1471/2896] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:52:34 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:52:34.482139 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f24e9-289f-4f4d-bd9e-fb9c21960d1a None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:34.483084 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f24e9-289f-4f4d-bd9e-fb9c21960d1a None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:34.483546 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f24e9-289f-4f4d-bd9e-fb9c21960d1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:34.488871 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a24d8f-45f5-49a9-ac64-d1a182f5449b None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:34.492690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-62a24d8f-45f5-49a9-ac64-d1a182f5449b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:34.502290 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-b47f24e9-289f-4f4d-bd9e-fb9c21960d1a None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:52:34.503509 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-b47f24e9-289f-4f4d-bd9e-fb9c21960d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:34.568012 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a24d8f-45f5-49a9-ac64-d1a182f5449b None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:34.578124 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1426/2897] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:52:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:52:34.588612 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f24e9-289f-4f4d-bd9e-fb9c21960d1a None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:34.591126 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-b47f24e9-289f-4f4d-bd9e-fb9c21960d1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:34.595913 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:52:34.631268 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36852ed-c5a5-4c4b-b1c5-72a2c39329b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:34.644315 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36852ed-c5a5-4c4b-b1c5-72a2c39329b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:34.644315 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36852ed-c5a5-4c4b-b1c5-72a2c39329b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:34.644315 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36852ed-c5a5-4c4b-b1c5-72a2c39329b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:34.662719 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f24e9-289f-4f4d-bd9e-fb9c21960d1a None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:34.666521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a16c2-2ac8-4869-9cf6-7072353f5a48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:34.667180 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a16c2-2ac8-4869-9cf6-7072353f5a48 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:34.667470 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a16c2-2ac8-4869-9cf6-7072353f5a48 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:34.668051 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a16c2-2ac8-4869-9cf6-7072353f5a48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:34.679258 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a16c2-2ac8-4869-9cf6-7072353f5a48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:34.684383 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a16c2-2ac8-4869-9cf6-7072353f5a48 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:34.691308 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1427/2898] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:52:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 17:52:34.694145 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36852ed-c5a5-4c4b-b1c5-72a2c39329b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:34.695620 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36852ed-c5a5-4c4b-b1c5-72a2c39329b8 service cinder] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:34.705246 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1472/2899] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:52:34 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:52:40.840420 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6de3fb-528b-4791-a649-41d50ded0d7b None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:40.840888 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6de3fb-528b-4791-a649-41d50ded0d7b None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:40.840976 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6de3fb-528b-4791-a649-41d50ded0d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:40.877057 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-4a6de3fb-528b-4791-a649-41d50ded0d7b None None] MFA Rules not processed for user `d2c4d7dd66614e52a006d1c6fb9d2fec`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:52:40.877969 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4a6de3fb-528b-4791-a649-41d50ded0d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:40.954047 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1428/2900] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:52:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:52:40.958532 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36852ed-c5a5-4c4b-b1c5-72a2c39329b8 service cinder] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:40.970466 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36852ed-c5a5-4c4b-b1c5-72a2c39329b8 service cinder] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:40.974996 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86657449-4438-476b-94c8-da3a8e9004b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:40.975659 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86657449-4438-476b-94c8-da3a8e9004b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:40.975972 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86657449-4438-476b-94c8-da3a8e9004b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:40.976306 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86657449-4438-476b-94c8-da3a8e9004b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:40.977004 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86657449-4438-476b-94c8-da3a8e9004b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:40.978903 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86657449-4438-476b-94c8-da3a8e9004b6 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:41.070901 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1473/2901] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:52:40 2022] POST /identity/v3/projects => generated 379 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 06 17:52:41.075076 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6de3fb-528b-4791-a649-41d50ded0d7b None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:41.087082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6de3fb-528b-4791-a649-41d50ded0d7b None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:41.101309 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:52:41.165877 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d70180-6a42-499c-9337-e16a4d534b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:41.166694 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d70180-6a42-499c-9337-e16a4d534b0e admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:41.167081 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d70180-6a42-499c-9337-e16a4d534b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:41.168301 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d70180-6a42-499c-9337-e16a4d534b0e admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:41.168898 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d70180-6a42-499c-9337-e16a4d534b0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:41.171826 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d70180-6a42-499c-9337-e16a4d534b0e admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:41.275399 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1429/2902] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:52:41 2022] POST /identity/v3/users => generated 296 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:52:41.279716 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86657449-4438-476b-94c8-da3a8e9004b6 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:41.290708 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86657449-4438-476b-94c8-da3a8e9004b6 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:41.296005 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8147e-f263-4621-99df-a1ed83ab1ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:41.296683 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8147e-f263-4621-99df-a1ed83ab1ecb admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:41.297020 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8147e-f263-4621-99df-a1ed83ab1ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:41.297823 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf8147e-f263-4621-99df-a1ed83ab1ecb admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:41.298558 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf8147e-f263-4621-99df-a1ed83ab1ecb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:41.300225 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf8147e-f263-4621-99df-a1ed83ab1ecb admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:41.311080 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1474/2903] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:52:41.321611 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d70180-6a42-499c-9337-e16a4d534b0e admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:41.331572 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d70180-6a42-499c-9337-e16a4d534b0e admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:41.336358 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166165b9-ca96-44a0-85a0-aa819994f917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:41.337272 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166165b9-ca96-44a0-85a0-aa819994f917 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:41.337554 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166165b9-ca96-44a0-85a0-aa819994f917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:41.337829 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166165b9-ca96-44a0-85a0-aa819994f917 admin admin] PATH_INFO: `/v3/projects/1f649aa2aae043788c8baf23c02d4414/users/f15f7f670eef426f8d12a0ea10c5b3de/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:41.353334 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166165b9-ca96-44a0-85a0-aa819994f917 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f649aa2aae043788c8baf23c02d4414, user_id=f15f7f670eef426f8d12a0ea10c5b3de, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:41.356002 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166165b9-ca96-44a0-85a0-aa819994f917 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:41.443544 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1430/2904] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:52:41 2022] PUT /identity/v3/projects/1f649aa2aae043788c8baf23c02d4414/users/f15f7f670eef426f8d12a0ea10c5b3de/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:52:41.448240 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8147e-f263-4621-99df-a1ed83ab1ecb admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:41.459162 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf8147e-f263-4621-99df-a1ed83ab1ecb admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:41.561630 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e737889a-6273-476d-984e-e8e71b7cb6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:41.562065 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737889a-6273-476d-984e-e8e71b7cb6cd admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:41.562357 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737889a-6273-476d-984e-e8e71b7cb6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:41.562662 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e737889a-6273-476d-984e-e8e71b7cb6cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:41.563398 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e737889a-6273-476d-984e-e8e71b7cb6cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:41.565298 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e737889a-6273-476d-984e-e8e71b7cb6cd admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:41.579402 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1475/2905] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:52:41 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:52:41.583447 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166165b9-ca96-44a0-85a0-aa819994f917 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:41.595272 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166165b9-ca96-44a0-85a0-aa819994f917 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:41.600231 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c2e7c-501d-4ce9-89b0-49c96008ed36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:41.601154 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c2e7c-501d-4ce9-89b0-49c96008ed36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:41.602402 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c2e7c-501d-4ce9-89b0-49c96008ed36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:41.602402 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c2e7c-501d-4ce9-89b0-49c96008ed36 admin admin] PATH_INFO: `/v3/projects/1f649aa2aae043788c8baf23c02d4414/users/f15f7f670eef426f8d12a0ea10c5b3de/roles/b41f8a7a58e64f749f2099ca45cc04a7` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:41.604082 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9c2e7c-501d-4ce9-89b0-49c96008ed36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f649aa2aae043788c8baf23c02d4414, user_id=f15f7f670eef426f8d12a0ea10c5b3de, role_id=b41f8a7a58e64f749f2099ca45cc04a7)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:41.607119 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9c2e7c-501d-4ce9-89b0-49c96008ed36 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:41.620942 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1431/2906] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:52:41 2022] PUT /identity/v3/projects/1f649aa2aae043788c8baf23c02d4414/users/f15f7f670eef426f8d12a0ea10c5b3de/roles/b41f8a7a58e64f749f2099ca45cc04a7 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 17:52:47.267687 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbcd293-e90b-499e-a77e-02a42cba8d31 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:47.268192 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbcd293-e90b-499e-a77e-02a42cba8d31 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:47.268341 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbcd293-e90b-499e-a77e-02a42cba8d31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:47.296877 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-5cbcd293-e90b-499e-a77e-02a42cba8d31 None None] MFA Rules not processed for user `f15f7f670eef426f8d12a0ea10c5b3de`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:52:47.298603 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-5cbcd293-e90b-499e-a77e-02a42cba8d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:47.329738 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:52:47.375464 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1476/2907] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:52:47 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:52:47.390240 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c2e7c-501d-4ce9-89b0-49c96008ed36 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:47.393945 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-db9c2e7c-501d-4ce9-89b0-49c96008ed36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:47.484128 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c2e7c-501d-4ce9-89b0-49c96008ed36 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:47.489229 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b3b26d-8b04-4c35-a29c-a1f08bb44a44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:47.489893 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3b26d-8b04-4c35-a29c-a1f08bb44a44 service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:47.490178 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3b26d-8b04-4c35-a29c-a1f08bb44a44 service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:47.490454 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b3b26d-8b04-4c35-a29c-a1f08bb44a44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:47.554979 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b3b26d-8b04-4c35-a29c-a1f08bb44a44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:47.556833 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b3b26d-8b04-4c35-a29c-a1f08bb44a44 service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:47.566212 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1432/2908] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:52:47 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:52:50.265089 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbcd293-e90b-499e-a77e-02a42cba8d31 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:52:50.267172 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-5cbcd293-e90b-499e-a77e-02a42cba8d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:50.415096 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbcd293-e90b-499e-a77e-02a42cba8d31 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:52:50.420869 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb344b-b6a7-488c-8519-50d45cc91b65 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76ca877bee9041d28ad41e45ef3dd18c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:52:50.421542 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb344b-b6a7-488c-8519-50d45cc91b65 service placement] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:52:50.422023 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb344b-b6a7-488c-8519-50d45cc91b65 service placement] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:52:50.422306 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdb344b-b6a7-488c-8519-50d45cc91b65 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:52:50.424601 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-bcdb344b-b6a7-488c-8519-50d45cc91b65 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:52:50.438889 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdb344b-b6a7-488c-8519-50d45cc91b65 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:52:50.440540 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdb344b-b6a7-488c-8519-50d45cc91b65 service placement] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:52:50.458159 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1477/2909] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:52:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:52:57.561975 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=92,keystone:INSERT=5 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:53:00.447233 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=48 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:53:07.504558 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac72fef-0ab5-44c8-ba7a-48d9f0a6b148 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:53:07.505030 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac72fef-0ab5-44c8-ba7a-48d9f0a6b148 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:53:07.505114 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac72fef-0ab5-44c8-ba7a-48d9f0a6b148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:53:07.562385 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-fac72fef-0ab5-44c8-ba7a-48d9f0a6b148 None None] MFA Rules not processed for user `1905e6f8c3e24e3cbfefce2827baa370`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:53:07.563589 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-fac72fef-0ab5-44c8-ba7a-48d9f0a6b148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:53:07.628168 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1433/2910] 10.222.0.40 () {58 vars in 1009 bytes} [Wed Jul 6 17:53:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:53:07.768926 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb344b-b6a7-488c-8519-50d45cc91b65 service placement] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:53:07.770404 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-bcdb344b-b6a7-488c-8519-50d45cc91b65 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:53:07.861500 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdb344b-b6a7-488c-8519-50d45cc91b65 service placement] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:53:07.869446 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f5dcb-32eb-486d-98bf-62cb4bbbb7bc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75acacd30e9a4cf2ad58b072b49e3f05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:53:07.870780 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f5dcb-32eb-486d-98bf-62cb4bbbb7bc service swift] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:53:07.871630 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f5dcb-32eb-486d-98bf-62cb4bbbb7bc service swift] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:53:07.872060 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178f5dcb-32eb-486d-98bf-62cb4bbbb7bc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:53:07.881218 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178f5dcb-32eb-486d-98bf-62cb4bbbb7bc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:53:07.883025 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178f5dcb-32eb-486d-98bf-62cb4bbbb7bc service swift] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:53:07.890889 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1478/2911] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 6 17:53:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 17:53:16.832986 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d269e7-7700-42a0-8993-006109a97edd None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:53:16.833557 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d269e7-7700-42a0-8993-006109a97edd None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:53:16.833651 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d269e7-7700-42a0-8993-006109a97edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:53:16.853363 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-25d269e7-7700-42a0-8993-006109a97edd None None] MFA Rules not processed for user `d2c4d7dd66614e52a006d1c6fb9d2fec`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:53:16.854575 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-25d269e7-7700-42a0-8993-006109a97edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:53:16.861888 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1434/2912] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:53:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:53:17.888406 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=41 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:53:25.006626 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f5dcb-32eb-486d-98bf-62cb4bbbb7bc service swift] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:53:25.097988 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178f5dcb-32eb-486d-98bf-62cb4bbbb7bc service swift] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:53:25.103506 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e85990-30ac-416e-b044-e715331536da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:53:25.104432 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e85990-30ac-416e-b044-e715331536da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:53:25.104713 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e85990-30ac-416e-b044-e715331536da admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:53:25.104992 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e85990-30ac-416e-b044-e715331536da admin admin] PATH_INFO: `/v3/users/c1c1057cc8e04cefacda126daf32babf` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:53:25.106195 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e85990-30ac-416e-b044-e715331536da admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1c1057cc8e04cefacda126daf32babf)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:53:25.108211 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e85990-30ac-416e-b044-e715331536da admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:53:25.351067 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-79e85990-30ac-416e-b044-e715331536da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1c1057cc8e04cefacda126daf32babf'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:53:25.370938 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-79e85990-30ac-416e-b044-e715331536da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1c1057cc8e04cefacda126daf32babf'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:53:25.371224 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-79e85990-30ac-416e-b044-e715331536da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1c1057cc8e04cefacda126daf32babf'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:53:25.387931 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-79e85990-30ac-416e-b044-e715331536da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1c1057cc8e04cefacda126daf32babf'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:53:25.584257 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-79e85990-30ac-416e-b044-e715331536da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1c1057cc8e04cefacda126daf32babf'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:53:25.590630 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1479/2913] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:53:25 2022] DELETE /identity/v3/users/c1c1057cc8e04cefacda126daf32babf => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:53:25.595145 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d94d5b-5df0-44b0-a7d6-0820918605e5 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:53:25.595637 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d94d5b-5df0-44b0-a7d6-0820918605e5 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:53:25.595752 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d94d5b-5df0-44b0-a7d6-0820918605e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:53:25.633215 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-11d94d5b-5df0-44b0-a7d6-0820918605e5 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:53:25.634476 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-11d94d5b-5df0-44b0-a7d6-0820918605e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:53:25.710008 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1435/2914] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:53:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:53:25.722316 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e85990-30ac-416e-b044-e715331536da admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:53:25.723854 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-79e85990-30ac-416e-b044-e715331536da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:53:25.811690 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e85990-30ac-416e-b044-e715331536da admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:53:25.816035 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f5f18-be84-4c48-92f5-69bfef6ac241 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:53:25.816702 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f5f18-be84-4c48-92f5-69bfef6ac241 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:53:25.817024 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f5f18-be84-4c48-92f5-69bfef6ac241 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:53:25.817258 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f5f18-be84-4c48-92f5-69bfef6ac241 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:53:25.823929 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:53:25.828287 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7f5f18-be84-4c48-92f5-69bfef6ac241 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:53:25.829936 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7f5f18-be84-4c48-92f5-69bfef6ac241 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:53:25.842462 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1480/2915] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:53:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:53:26.287811 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d94d5b-5df0-44b0-a7d6-0820918605e5 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:53:26.287811 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-11d94d5b-5df0-44b0-a7d6-0820918605e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:53:26.298148 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d94d5b-5df0-44b0-a7d6-0820918605e5 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:53:26.304795 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160aa98a-5bbd-4699-a2eb-bb314bef3029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:53:26.305732 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160aa98a-5bbd-4699-a2eb-bb314bef3029 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:53:26.306022 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160aa98a-5bbd-4699-a2eb-bb314bef3029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:53:26.306303 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160aa98a-5bbd-4699-a2eb-bb314bef3029 admin admin] PATH_INFO: `/v3/projects/657f67d43f59480e82347efca1023898` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:53:26.307664 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160aa98a-5bbd-4699-a2eb-bb314bef3029 admin admin] RBAC: Authorizing `identity:delete_project(project_id=657f67d43f59480e82347efca1023898)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:53:26.309618 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160aa98a-5bbd-4699-a2eb-bb314bef3029 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:53:26.433171 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:53:26.443772 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-160aa98a-5bbd-4699-a2eb-bb314bef3029 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '657f67d43f59480e82347efca1023898', 'request_id': 'req-160aa98a-5bbd-4699-a2eb-bb314bef3029'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:53:26.450094 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-160aa98a-5bbd-4699-a2eb-bb314bef3029 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '657f67d43f59480e82347efca1023898', 'request_id': 'req-160aa98a-5bbd-4699-a2eb-bb314bef3029'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:53:26.532520 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-160aa98a-5bbd-4699-a2eb-bb314bef3029 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '657f67d43f59480e82347efca1023898', 'request_id': 'req-160aa98a-5bbd-4699-a2eb-bb314bef3029'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:53:26.546354 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-160aa98a-5bbd-4699-a2eb-bb314bef3029 admin admin] The token cache is being invalidate because project 657f67d43f59480e82347efca1023898 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:53:26.547291 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-160aa98a-5bbd-4699-a2eb-bb314bef3029 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:53:26.548053 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-160aa98a-5bbd-4699-a2eb-bb314bef3029 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:53:26.556874 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1436/2916] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:53:26 2022] DELETE /identity/v3/projects/657f67d43f59480e82347efca1023898 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:53:28.150075 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10b24c-5d9c-4d16-8661-7500c63bdb62 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:53:28.150487 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10b24c-5d9c-4d16-8661-7500c63bdb62 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:53:28.150645 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e10b24c-5d9c-4d16-8661-7500c63bdb62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:53:28.168014 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-9e10b24c-5d9c-4d16-8661-7500c63bdb62 None None] MFA Rules not processed for user `1905e6f8c3e24e3cbfefce2827baa370`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:53:28.169459 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-9e10b24c-5d9c-4d16-8661-7500c63bdb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:53:28.223113 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1481/2917] 10.222.0.40 () {58 vars in 967 bytes} [Wed Jul 6 17:53:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:53:28.235175 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160aa98a-5bbd-4699-a2eb-bb314bef3029 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:53:28.236524 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-160aa98a-5bbd-4699-a2eb-bb314bef3029 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:53:28.308565 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160aa98a-5bbd-4699-a2eb-bb314bef3029 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:53:28.313462 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a0edaa-b41c-485b-8b1d-7f15a9d3a541 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '75acacd30e9a4cf2ad58b072b49e3f05', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:53:28.314106 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0edaa-b41c-485b-8b1d-7f15a9d3a541 service swift] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:53:28.314389 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0edaa-b41c-485b-8b1d-7f15a9d3a541 service swift] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:53:28.314665 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a0edaa-b41c-485b-8b1d-7f15a9d3a541 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:53:28.323166 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a0edaa-b41c-485b-8b1d-7f15a9d3a541 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:53:28.324772 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a0edaa-b41c-485b-8b1d-7f15a9d3a541 service swift] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:53:28.332559 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1437/2918] 10.222.0.40 () {60 vars in 1384 bytes} [Wed Jul 6 17:53:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 06 17:53:31.372823 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e10b24c-5d9c-4d16-8661-7500c63bdb62 None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:53:31.374268 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-9e10b24c-5d9c-4d16-8661-7500c63bdb62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:53:31.446014 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e10b24c-5d9c-4d16-8661-7500c63bdb62 None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:53:31.451054 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dcb1ae-4cbc-4c3a-bf03-d9947fb01851 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76ca877bee9041d28ad41e45ef3dd18c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:53:31.451703 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dcb1ae-4cbc-4c3a-bf03-d9947fb01851 service placement] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:53:31.452021 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dcb1ae-4cbc-4c3a-bf03-d9947fb01851 service placement] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:53:31.452298 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dcb1ae-4cbc-4c3a-bf03-d9947fb01851 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:53:31.454076 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-65dcb1ae-4cbc-4c3a-bf03-d9947fb01851 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:53:31.466588 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dcb1ae-4cbc-4c3a-bf03-d9947fb01851 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:53:31.468249 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dcb1ae-4cbc-4c3a-bf03-d9947fb01851 service placement] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:53:31.478714 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1482/2919] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:53:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:53:31.509773 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a0edaa-b41c-485b-8b1d-7f15a9d3a541 service swift] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:53:31.511221 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-10a0edaa-b41c-485b-8b1d-7f15a9d3a541 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:53:31.577605 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a0edaa-b41c-485b-8b1d-7f15a9d3a541 service swift] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:53:31.582099 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea5c015-769c-48ea-abb7-38d64f942816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:53:31.582974 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea5c015-769c-48ea-abb7-38d64f942816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:53:31.583269 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea5c015-769c-48ea-abb7-38d64f942816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:53:31.583545 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea5c015-769c-48ea-abb7-38d64f942816 admin admin] PATH_INFO: `/v3/users/f437559c9a344df7bde785d5e494516b` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:53:31.584764 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea5c015-769c-48ea-abb7-38d64f942816 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f437559c9a344df7bde785d5e494516b)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:53:31.586673 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea5c015-769c-48ea-abb7-38d64f942816 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:53:31.762655 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-aea5c015-769c-48ea-abb7-38d64f942816 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f437559c9a344df7bde785d5e494516b'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:53:31.767845 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:53:31.774956 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-aea5c015-769c-48ea-abb7-38d64f942816 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f437559c9a344df7bde785d5e494516b'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:53:31.781886 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-aea5c015-769c-48ea-abb7-38d64f942816 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f437559c9a344df7bde785d5e494516b'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:53:31.782611 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-aea5c015-769c-48ea-abb7-38d64f942816 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f437559c9a344df7bde785d5e494516b'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:53:31.852133 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-aea5c015-769c-48ea-abb7-38d64f942816 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f437559c9a344df7bde785d5e494516b'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:53:31.858269 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1438/2920] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:53:31 2022] DELETE /identity/v3/users/f437559c9a344df7bde785d5e494516b => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:53:32.105356 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dcb1ae-4cbc-4c3a-bf03-d9947fb01851 service placement] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:53:32.115783 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dcb1ae-4cbc-4c3a-bf03-d9947fb01851 service placement] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:53:32.172074 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121a2704-0c91-422d-961c-f36a0693be6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:53:32.172932 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a2704-0c91-422d-961c-f36a0693be6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:53:32.173217 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a2704-0c91-422d-961c-f36a0693be6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:53:32.173490 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121a2704-0c91-422d-961c-f36a0693be6a admin admin] PATH_INFO: `/v3/projects/387788bc8d4e4afab5a34f746f475983` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:53:32.174669 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121a2704-0c91-422d-961c-f36a0693be6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=387788bc8d4e4afab5a34f746f475983)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:53:32.176618 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121a2704-0c91-422d-961c-f36a0693be6a admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:53:32.257314 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:53:32.270228 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-121a2704-0c91-422d-961c-f36a0693be6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '387788bc8d4e4afab5a34f746f475983', 'request_id': 'req-121a2704-0c91-422d-961c-f36a0693be6a'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:53:32.276944 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-121a2704-0c91-422d-961c-f36a0693be6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '387788bc8d4e4afab5a34f746f475983', 'request_id': 'req-121a2704-0c91-422d-961c-f36a0693be6a'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:53:32.282349 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-121a2704-0c91-422d-961c-f36a0693be6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '387788bc8d4e4afab5a34f746f475983', 'request_id': 'req-121a2704-0c91-422d-961c-f36a0693be6a'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:53:32.338028 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-121a2704-0c91-422d-961c-f36a0693be6a admin admin] The token cache is being invalidate because project 387788bc8d4e4afab5a34f746f475983 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80469) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:53:32.338323 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-121a2704-0c91-422d-961c-f36a0693be6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:53:32.339021 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-121a2704-0c91-422d-961c-f36a0693be6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:53:32.343550 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1483/2921] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:53:32 2022] DELETE /identity/v3/projects/387788bc8d4e4afab5a34f746f475983 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:53:32.349370 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718dc940-6883-44ed-b2a1-93f56fd9d712 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:53:32.349370 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718dc940-6883-44ed-b2a1-93f56fd9d712 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:53:32.349370 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718dc940-6883-44ed-b2a1-93f56fd9d712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:53:32.394947 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-718dc940-6883-44ed-b2a1-93f56fd9d712 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:53:32.396776 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-718dc940-6883-44ed-b2a1-93f56fd9d712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:53:32.456076 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1439/2922] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:53:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 06 17:53:32.464643 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f52d1a-da2d-4b6a-ab78-ed0fc8e473ee None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:53:32.465167 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f52d1a-da2d-4b6a-ab78-ed0fc8e473ee None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:53:32.465477 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f52d1a-da2d-4b6a-ab78-ed0fc8e473ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:53:32.484636 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-49f52d1a-da2d-4b6a-ab78-ed0fc8e473ee None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:53:32.485896 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-49f52d1a-da2d-4b6a-ab78-ed0fc8e473ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:53:32.494402 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1484/2923] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:53:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:53:32.498672 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718dc940-6883-44ed-b2a1-93f56fd9d712 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:53:32.509281 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718dc940-6883-44ed-b2a1-93f56fd9d712 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:53:32.514115 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd90d823-4bfe-448c-9ab8-0c3249cc0a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:53:32.514775 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd90d823-4bfe-448c-9ab8-0c3249cc0a0a admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:53:32.515062 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd90d823-4bfe-448c-9ab8-0c3249cc0a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:53:32.515387 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd90d823-4bfe-448c-9ab8-0c3249cc0a0a admin admin] PATH_INFO: `/v3/domains` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:53:32.516180 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd90d823-4bfe-448c-9ab8-0c3249cc0a0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80470) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 06 17:53:32.516490 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd90d823-4bfe-448c-9ab8-0c3249cc0a0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:53:32.518107 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd90d823-4bfe-448c-9ab8-0c3249cc0a0a admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:53:32.529612 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1440/2924] 10.222.0.40 () {58 vars in 1152 bytes} [Wed Jul 6 17:53:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 06 17:53:32.533865 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f52d1a-da2d-4b6a-ab78-ed0fc8e473ee None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:53:32.543652 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f52d1a-da2d-4b6a-ab78-ed0fc8e473ee None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:53:32.548580 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8681c-151f-45d7-87f4-c7f7db0a16c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:53:32.549222 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8681c-151f-45d7-87f4-c7f7db0a16c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:53:32.549504 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8681c-151f-45d7-87f4-c7f7db0a16c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:53:32.549779 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8681c-151f-45d7-87f4-c7f7db0a16c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:53:32.550471 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f8681c-151f-45d7-87f4-c7f7db0a16c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:53:32.552970 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f8681c-151f-45d7-87f4-c7f7db0a16c4 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:53:32.653584 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1485/2925] 10.222.0.40 () {60 vars in 1152 bytes} [Wed Jul 6 17:53:32 2022] POST /identity/v3/projects => generated 367 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:53:32.657765 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd90d823-4bfe-448c-9ab8-0c3249cc0a0a admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:53:32.667697 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd90d823-4bfe-448c-9ab8-0c3249cc0a0a admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:53:32.723240 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d4653-ccd2-419a-9605-503a5f9e08bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:53:32.723941 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d4653-ccd2-419a-9605-503a5f9e08bb admin admin] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:53:32.724262 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d4653-ccd2-419a-9605-503a5f9e08bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:53:32.724546 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24d4653-ccd2-419a-9605-503a5f9e08bb admin admin] PATH_INFO: `/v3/users` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:53:32.725246 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d4653-ccd2-419a-9605-503a5f9e08bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:53:32.727229 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24d4653-ccd2-419a-9605-503a5f9e08bb admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:53:32.795221 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1441/2926] 10.222.0.40 () {60 vars in 1143 bytes} [Wed Jul 6 17:53:32 2022] POST /identity/v3/users => generated 284 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 06 17:53:32.799343 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8681c-151f-45d7-87f4-c7f7db0a16c4 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:53:32.810098 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8681c-151f-45d7-87f4-c7f7db0a16c4 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:53:32.814965 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815b518-dfc1-4c89-852e-0cc3320b36bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:53:32.815622 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815b518-dfc1-4c89-852e-0cc3320b36bc admin admin] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:53:32.815938 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815b518-dfc1-4c89-852e-0cc3320b36bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:53:32.816215 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815b518-dfc1-4c89-852e-0cc3320b36bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:53:32.816937 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6815b518-dfc1-4c89-852e-0cc3320b36bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:53:32.819099 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6815b518-dfc1-4c89-852e-0cc3320b36bc admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:53:32.828862 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1486/2927] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:53:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 06 17:53:32.832792 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d4653-ccd2-419a-9605-503a5f9e08bb admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:53:32.842559 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24d4653-ccd2-419a-9605-503a5f9e08bb admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:53:32.847112 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ec1e73-e741-431e-8e02-e1d2332b2349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:53:32.847789 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec1e73-e741-431e-8e02-e1d2332b2349 admin admin] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:53:32.848074 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec1e73-e741-431e-8e02-e1d2332b2349 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:53:32.848348 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ec1e73-e741-431e-8e02-e1d2332b2349 admin admin] PATH_INFO: `/v3/roles` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:53:32.849065 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec1e73-e741-431e-8e02-e1d2332b2349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:53:32.850636 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ec1e73-e741-431e-8e02-e1d2332b2349 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:53:32.860322 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1442/2928] 10.222.0.40 () {58 vars in 1121 bytes} [Wed Jul 6 17:53:32 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 06 17:53:32.864159 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815b518-dfc1-4c89-852e-0cc3320b36bc admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:53:32.874311 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815b518-dfc1-4c89-852e-0cc3320b36bc admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:53:32.878848 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e203f7-54f0-409a-b53f-f0ca6be7ef62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:53:32.879754 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e203f7-54f0-409a-b53f-f0ca6be7ef62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:53:32.880048 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e203f7-54f0-409a-b53f-f0ca6be7ef62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:53:32.880372 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e203f7-54f0-409a-b53f-f0ca6be7ef62 admin admin] PATH_INFO: `/v3/projects/e60e91d0ac9c477f99b782c6f803e685/users/ad8a359ad7ae4e479f95391d6ee27a72/roles/7fd63d2d99254c00bb3501173643ee30` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:53:32.895176 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e203f7-54f0-409a-b53f-f0ca6be7ef62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e60e91d0ac9c477f99b782c6f803e685, user_id=ad8a359ad7ae4e479f95391d6ee27a72, role_id=7fd63d2d99254c00bb3501173643ee30)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:53:32.897814 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e203f7-54f0-409a-b53f-f0ca6be7ef62 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:53:32.949452 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1487/2929] 10.222.0.40 () {60 vars in 1482 bytes} [Wed Jul 6 17:53:32 2022] PUT /identity/v3/projects/e60e91d0ac9c477f99b782c6f803e685/users/ad8a359ad7ae4e479f95391d6ee27a72/roles/7fd63d2d99254c00bb3501173643ee30 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:53:32.959167 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4891fe-1852-4f1e-9555-4681b8e4e4f7 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:53:32.961897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4891fe-1852-4f1e-9555-4681b8e4e4f7 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:53:32.961897 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4891fe-1852-4f1e-9555-4681b8e4e4f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:53:32.983294 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:53:32.986047 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-8f4891fe-1852-4f1e-9555-4681b8e4e4f7 None None] MFA Rules not processed for user `ad8a359ad7ae4e479f95391d6ee27a72`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:53:32.987256 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-8f4891fe-1852-4f1e-9555-4681b8e4e4f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:53:33.050521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1443/2930] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:53:32 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:53:33.062491 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e203f7-54f0-409a-b53f-f0ca6be7ef62 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:53:33.064251 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-29e203f7-54f0-409a-b53f-f0ca6be7ef62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:53:33.142196 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e203f7-54f0-409a-b53f-f0ca6be7ef62 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:53:33.145922 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d6af6-ed2b-4372-acee-eb8a78894eb0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '635a410f471c43109438d0c39874070b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:53:33.146562 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d6af6-ed2b-4372-acee-eb8a78894eb0 service glance] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:53:33.146843 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d6af6-ed2b-4372-acee-eb8a78894eb0 service glance] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:53:33.147123 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d6af6-ed2b-4372-acee-eb8a78894eb0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:53:33.155571 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35d6af6-ed2b-4372-acee-eb8a78894eb0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:53:33.157020 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35d6af6-ed2b-4372-acee-eb8a78894eb0 service glance] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:53:33.166322 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1488/2931] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:53:33 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:53:38.536680 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4891fe-1852-4f1e-9555-4681b8e4e4f7 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:53:38.538110 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-8f4891fe-1852-4f1e-9555-4681b8e4e4f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:53:38.599973 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4891fe-1852-4f1e-9555-4681b8e4e4f7 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:53:38.603549 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2752de-cec3-4909-9cd0-eb0aa70e6a92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:53:38.604235 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2752de-cec3-4909-9cd0-eb0aa70e6a92 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:53:38.604521 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2752de-cec3-4909-9cd0-eb0aa70e6a92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:53:38.604795 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2752de-cec3-4909-9cd0-eb0aa70e6a92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:53:38.606586 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-4b2752de-cec3-4909-9cd0-eb0aa70e6a92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:53:38.795322 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2752de-cec3-4909-9cd0-eb0aa70e6a92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:53:38.797272 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2752de-cec3-4909-9cd0-eb0aa70e6a92 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:53:38.809269 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1444/2932] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:53:38 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:53:43.161782 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=79,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:53:48.804541 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=76 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:54:09.230412 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d6af6-ed2b-4372-acee-eb8a78894eb0 service glance] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:54:09.295792 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d6af6-ed2b-4372-acee-eb8a78894eb0 service glance] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:54:09.301120 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba21666-651f-4798-af24-c67e08e67243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:54:09.302008 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba21666-651f-4798-af24-c67e08e67243 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:54:09.302295 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba21666-651f-4798-af24-c67e08e67243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:54:09.302570 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba21666-651f-4798-af24-c67e08e67243 admin admin] PATH_INFO: `/v3/users/ad8a359ad7ae4e479f95391d6ee27a72` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:54:09.303784 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba21666-651f-4798-af24-c67e08e67243 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad8a359ad7ae4e479f95391d6ee27a72)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:54:09.305708 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba21666-651f-4798-af24-c67e08e67243 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:54:09.518072 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-6ba21666-651f-4798-af24-c67e08e67243 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad8a359ad7ae4e479f95391d6ee27a72'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:54:09.524936 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-6ba21666-651f-4798-af24-c67e08e67243 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad8a359ad7ae4e479f95391d6ee27a72'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:54:09.531034 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-6ba21666-651f-4798-af24-c67e08e67243 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad8a359ad7ae4e479f95391d6ee27a72'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:54:09.538929 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-6ba21666-651f-4798-af24-c67e08e67243 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad8a359ad7ae4e479f95391d6ee27a72'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:54:09.652712 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-6ba21666-651f-4798-af24-c67e08e67243 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad8a359ad7ae4e479f95391d6ee27a72'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:54:09.659819 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1489/2933] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:54:09 2022] DELETE /identity/v3/users/ad8a359ad7ae4e479f95391d6ee27a72 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:54:09.664401 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0a35e-7ceb-4f5d-8bda-770913027923 None None] REQUEST_METHOD: `POST` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:54:09.664874 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0a35e-7ceb-4f5d-8bda-770913027923 None None] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:54:09.664970 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0a35e-7ceb-4f5d-8bda-770913027923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:54:09.682014 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.auth.core [None req-b1e0a35e-7ceb-4f5d-8bda-770913027923 None None] MFA Rules not processed for user `fb49b9dd86cd415bb41412ca15c4d0ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80470) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:54:09.683223 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-b1e0a35e-7ceb-4f5d-8bda-770913027923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:54:09.741452 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1445/2934] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:54:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:54:09.751111 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba21666-651f-4798-af24-c67e08e67243 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:54:09.752573 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-6ba21666-651f-4798-af24-c67e08e67243 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:54:09.810713 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba21666-651f-4798-af24-c67e08e67243 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:54:09.814160 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3962a91-a7bb-4580-9146-f796a395a38d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:54:09.814825 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3962a91-a7bb-4580-9146-f796a395a38d service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:54:09.815109 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3962a91-a7bb-4580-9146-f796a395a38d service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:54:09.815383 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3962a91-a7bb-4580-9146-f796a395a38d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:54:09.824091 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3962a91-a7bb-4580-9146-f796a395a38d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:54:09.825497 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3962a91-a7bb-4580-9146-f796a395a38d service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:54:09.835348 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1490/2935] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:54:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:54:10.277096 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e0a35e-7ceb-4f5d-8bda-770913027923 None None] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:54:10.278584 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-b1e0a35e-7ceb-4f5d-8bda-770913027923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:54:10.291202 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e0a35e-7ceb-4f5d-8bda-770913027923 None None] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:54:10.296142 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00822ee1-359e-42d4-b958-82ee6cdd2360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:54:10.297016 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00822ee1-359e-42d4-b958-82ee6cdd2360 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:54:10.297306 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00822ee1-359e-42d4-b958-82ee6cdd2360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:54:10.297583 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00822ee1-359e-42d4-b958-82ee6cdd2360 admin admin] PATH_INFO: `/v3/projects/e60e91d0ac9c477f99b782c6f803e685` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:54:10.298800 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00822ee1-359e-42d4-b958-82ee6cdd2360 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e60e91d0ac9c477f99b782c6f803e685)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:54:10.300891 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00822ee1-359e-42d4-b958-82ee6cdd2360 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:54:10.403146 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-00822ee1-359e-42d4-b958-82ee6cdd2360 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e60e91d0ac9c477f99b782c6f803e685', 'request_id': 'req-00822ee1-359e-42d4-b958-82ee6cdd2360'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:54:10.410107 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-00822ee1-359e-42d4-b958-82ee6cdd2360 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e60e91d0ac9c477f99b782c6f803e685', 'request_id': 'req-00822ee1-359e-42d4-b958-82ee6cdd2360'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:54:10.474928 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-00822ee1-359e-42d4-b958-82ee6cdd2360 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e60e91d0ac9c477f99b782c6f803e685', 'request_id': 'req-00822ee1-359e-42d4-b958-82ee6cdd2360'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:54:10.485934 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-00822ee1-359e-42d4-b958-82ee6cdd2360 admin admin] The token cache is being invalidate because project e60e91d0ac9c477f99b782c6f803e685 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:54:10.486253 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-00822ee1-359e-42d4-b958-82ee6cdd2360 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:54:10.486876 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-00822ee1-359e-42d4-b958-82ee6cdd2360 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:54:10.488441 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1446/2936] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:54:10 2022] DELETE /identity/v3/projects/e60e91d0ac9c477f99b782c6f803e685 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:54:19.831229 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:54:20.482813 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:55:43.973256 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e34596c-9da2-4780-8689-35af121b6249 None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:55:43.973742 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e34596c-9da2-4780-8689-35af121b6249 None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:55:43.973836 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e34596c-9da2-4780-8689-35af121b6249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:55:43.991082 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-9e34596c-9da2-4780-8689-35af121b6249 None None] MFA Rules not processed for user `d2c4d7dd66614e52a006d1c6fb9d2fec`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:55:43.992298 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-9e34596c-9da2-4780-8689-35af121b6249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:55:44.048008 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1491/2937] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:55:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:55:54.040909 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=25 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:56:04.721194 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00822ee1-359e-42d4-b958-82ee6cdd2360 admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:56:04.724630 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-00822ee1-359e-42d4-b958-82ee6cdd2360 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:56:04.802556 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00822ee1-359e-42d4-b958-82ee6cdd2360 admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:56:04.807978 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05084a33-7800-4580-8c44-73b3b3416e8e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:56:04.808632 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05084a33-7800-4580-8c44-73b3b3416e8e service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:56:04.808924 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05084a33-7800-4580-8c44-73b3b3416e8e service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:56:04.809201 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05084a33-7800-4580-8c44-73b3b3416e8e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:56:04.810995 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-05084a33-7800-4580-8c44-73b3b3416e8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:56:04.823031 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05084a33-7800-4580-8c44-73b3b3416e8e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:56:04.824702 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05084a33-7800-4580-8c44-73b3b3416e8e service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:56:04.835429 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1447/2938] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:56:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:56:14.830921 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=34 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:57:40.583991 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c38f2-dfa3-4bbd-91b7-225217406afb None None] REQUEST_METHOD: `POST` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:57:40.584488 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c38f2-dfa3-4bbd-91b7-225217406afb None None] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:57:40.584582 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c38f2-dfa3-4bbd-91b7-225217406afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:57:40.601926 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.auth.core [None req-9e0c38f2-dfa3-4bbd-91b7-225217406afb None None] MFA Rules not processed for user `d2c4d7dd66614e52a006d1c6fb9d2fec`. Rule list: `[]` (Enabled: `True`). {{(pid=80469) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 06 17:57:40.603120 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-9e0c38f2-dfa3-4bbd-91b7-225217406afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:57:40.609899 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1492/2939] 10.222.0.40 () {58 vars in 957 bytes} [Wed Jul 6 17:57:40 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 06 17:57:40.646367 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05084a33-7800-4580-8c44-73b3b3416e8e service nova] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:57:40.647762 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-05084a33-7800-4580-8c44-73b3b3416e8e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:57:40.715863 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05084a33-7800-4580-8c44-73b3b3416e8e service nova] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:57:40.719429 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86214220-8cc6-45e2-ac89-435f5f358921 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a1bfa3e856ae46d48c82195f70aec24f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:57:40.720113 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86214220-8cc6-45e2-ac89-435f5f358921 service cinder] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:57:40.720401 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86214220-8cc6-45e2-ac89-435f5f358921 service cinder] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:57:40.720708 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86214220-8cc6-45e2-ac89-435f5f358921 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:57:40.722563 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-86214220-8cc6-45e2-ac89-435f5f358921 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:57:40.736326 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86214220-8cc6-45e2-ac89-435f5f358921 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:57:40.737691 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86214220-8cc6-45e2-ac89-435f5f358921 service cinder] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:57:40.748583 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1448/2940] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:57:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:57:41.025632 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c38f2-dfa3-4bbd-91b7-225217406afb None None] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:57:41.027065 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-9e0c38f2-dfa3-4bbd-91b7-225217406afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:57:41.082533 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c38f2-dfa3-4bbd-91b7-225217406afb None None] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:57:41.086114 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5fe3f4-bc53-4e5d-ad72-0ef26f548070 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:57:41.086751 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5fe3f4-bc53-4e5d-ad72-0ef26f548070 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:57:41.087035 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5fe3f4-bc53-4e5d-ad72-0ef26f548070 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:57:41.087311 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5fe3f4-bc53-4e5d-ad72-0ef26f548070 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:57:41.089131 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-2c5fe3f4-bc53-4e5d-ad72-0ef26f548070 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:57:41.098374 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5fe3f4-bc53-4e5d-ad72-0ef26f548070 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:57:41.099814 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5fe3f4-bc53-4e5d-ad72-0ef26f548070 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:57:41.107773 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1493/2941] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:57:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:57:41.814923 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86214220-8cc6-45e2-ac89-435f5f358921 service cinder] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:57:41.816400 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-86214220-8cc6-45e2-ac89-435f5f358921 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:57:41.824700 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5fe3f4-bc53-4e5d-ad72-0ef26f548070 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:57:41.826119 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-2c5fe3f4-bc53-4e5d-ad72-0ef26f548070 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:57:41.836743 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5fe3f4-bc53-4e5d-ad72-0ef26f548070 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:57:41.840972 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8f1a3-dec3-4077-bdee-b56fe763fba4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:57:41.841609 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8f1a3-dec3-4077-bdee-b56fe763fba4 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:57:41.841888 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8f1a3-dec3-4077-bdee-b56fe763fba4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:57:41.842160 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e8f1a3-dec3-4077-bdee-b56fe763fba4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:57:41.844066 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-91e8f1a3-dec3-4077-bdee-b56fe763fba4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:57:41.858265 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e8f1a3-dec3-4077-bdee-b56fe763fba4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:57:41.864817 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e8f1a3-dec3-4077-bdee-b56fe763fba4 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:57:41.871416 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1494/2942] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:57:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 06 17:57:41.887238 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86214220-8cc6-45e2-ac89-435f5f358921 service cinder] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:57:41.892256 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f24bba-5c7d-497b-86e8-729b6745c317 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76ca877bee9041d28ad41e45ef3dd18c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:57:41.894945 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f24bba-5c7d-497b-86e8-729b6745c317 service placement] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:57:41.894945 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f24bba-5c7d-497b-86e8-729b6745c317 service placement] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:57:41.894945 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f24bba-5c7d-497b-86e8-729b6745c317 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:57:41.895622 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-19f24bba-5c7d-497b-86e8-729b6745c317 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:57:41.908084 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f24bba-5c7d-497b-86e8-729b6745c317 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:57:41.909684 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f24bba-5c7d-497b-86e8-729b6745c317 service placement] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:57:41.920535 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1449/2943] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:57:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:57:45.792441 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8f1a3-dec3-4077-bdee-b56fe763fba4 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:57:45.801649 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e8f1a3-dec3-4077-bdee-b56fe763fba4 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:57:45.805317 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13843b8-d199-4cd1-9072-b2fa7326b379 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:57:45.805953 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13843b8-d199-4cd1-9072-b2fa7326b379 service neutron] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:57:45.806267 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13843b8-d199-4cd1-9072-b2fa7326b379 service neutron] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:57:45.806595 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13843b8-d199-4cd1-9072-b2fa7326b379 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:57:45.808399 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d13843b8-d199-4cd1-9072-b2fa7326b379 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:57:45.817870 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13843b8-d199-4cd1-9072-b2fa7326b379 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:57:45.819313 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13843b8-d199-4cd1-9072-b2fa7326b379 service neutron] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:57:45.828284 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1495/2944] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:57:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:57:46.653111 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f24bba-5c7d-497b-86e8-729b6745c317 service placement] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:57:46.658825 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-19f24bba-5c7d-497b-86e8-729b6745c317 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:57:46.676497 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f24bba-5c7d-497b-86e8-729b6745c317 service placement] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:57:46.683335 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a0298-7966-47f0-895b-22657ee838f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:57:46.684056 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a0298-7966-47f0-895b-22657ee838f2 service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:57:46.684338 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a0298-7966-47f0-895b-22657ee838f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:57:46.684615 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a0298-7966-47f0-895b-22657ee838f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:57:46.686459 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-966a0298-7966-47f0-895b-22657ee838f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:57:46.699485 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966a0298-7966-47f0-895b-22657ee838f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:57:46.701151 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966a0298-7966-47f0-895b-22657ee838f2 service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:57:46.712472 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1450/2945] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:57:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:57:48.656894 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13843b8-d199-4cd1-9072-b2fa7326b379 service neutron] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:57:48.690043 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13843b8-d199-4cd1-9072-b2fa7326b379 service neutron] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:57:48.695613 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8145013-24e5-462c-b78a-24701826a74a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:57:48.696295 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8145013-24e5-462c-b78a-24701826a74a service nova] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:57:48.696586 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8145013-24e5-462c-b78a-24701826a74a service nova] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:57:48.696868 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8145013-24e5-462c-b78a-24701826a74a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:57:48.698663 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-f8145013-24e5-462c-b78a-24701826a74a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:57:48.719166 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8145013-24e5-462c-b78a-24701826a74a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:57:48.720797 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8145013-24e5-462c-b78a-24701826a74a service nova] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:57:48.730415 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1496/2946] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:57:48 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:57:56.707594 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=69 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:57:58.725694 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=60 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:57:58.860677 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a0298-7966-47f0-895b-22657ee838f2 service nova] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:57:58.890630 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a0298-7966-47f0-895b-22657ee838f2 service nova] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:57:58.896184 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ecf271-56bf-42bf-802a-04eea7ae57c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af26ef96dc9d47dc91549b736939268e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:57:58.896827 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ecf271-56bf-42bf-802a-04eea7ae57c5 service nova] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:57:58.897110 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ecf271-56bf-42bf-802a-04eea7ae57c5 service nova] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:57:58.897385 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ecf271-56bf-42bf-802a-04eea7ae57c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:57:58.899159 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-f0ecf271-56bf-42bf-802a-04eea7ae57c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:57:58.921982 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ecf271-56bf-42bf-802a-04eea7ae57c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:57:58.923611 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ecf271-56bf-42bf-802a-04eea7ae57c5 service nova] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:57:58.935932 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1451/2947] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:57:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:57:59.171546 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8145013-24e5-462c-b78a-24701826a74a service nova] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:57:59.193854 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8145013-24e5-462c-b78a-24701826a74a service nova] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:57:59.199036 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e355f8d-44ae-4de7-a64f-41a490b06a3e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76ca877bee9041d28ad41e45ef3dd18c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:57:59.199688 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e355f8d-44ae-4de7-a64f-41a490b06a3e service placement] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:57:59.200037 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e355f8d-44ae-4de7-a64f-41a490b06a3e service placement] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:57:59.200317 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e355f8d-44ae-4de7-a64f-41a490b06a3e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:57:59.202240 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-8e355f8d-44ae-4de7-a64f-41a490b06a3e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:57:59.229731 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e355f8d-44ae-4de7-a64f-41a490b06a3e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:57:59.231323 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e355f8d-44ae-4de7-a64f-41a490b06a3e service placement] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:57:59.242363 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1497/2948] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:57:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:58:08.930929 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=19 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:58:09.238146 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=15 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:59:48.217931 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ecf271-56bf-42bf-802a-04eea7ae57c5 service nova] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:59:48.227849 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ecf271-56bf-42bf-802a-04eea7ae57c5 service nova] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:59:48.231654 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2460c4-7881-4b20-9b1e-bc7e273649b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5f479458a68b4daf9f3a7ae53d6c3763', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:59:48.232326 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2460c4-7881-4b20-9b1e-bc7e273649b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:59:48.232611 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2460c4-7881-4b20-9b1e-bc7e273649b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:59:48.232911 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2460c4-7881-4b20-9b1e-bc7e273649b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:59:48.234660 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-cd2460c4-7881-4b20-9b1e-bc7e273649b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:59:48.302482 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2460c4-7881-4b20-9b1e-bc7e273649b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:59:48.304529 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2460c4-7881-4b20-9b1e-bc7e273649b7 service neutron] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:59:48.314969 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1452/2949] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:59:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:59:51.499565 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e355f8d-44ae-4de7-a64f-41a490b06a3e service placement] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:59:51.511542 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e355f8d-44ae-4de7-a64f-41a490b06a3e service placement] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:59:51.516709 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cba15b-69cf-485e-9a35-7663af661e15 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76ca877bee9041d28ad41e45ef3dd18c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:59:51.517357 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cba15b-69cf-485e-9a35-7663af661e15 service placement] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:59:51.517639 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cba15b-69cf-485e-9a35-7663af661e15 service placement] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:59:51.517912 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cba15b-69cf-485e-9a35-7663af661e15 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:59:51.519672 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-92cba15b-69cf-485e-9a35-7663af661e15 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:59:51.531387 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cba15b-69cf-485e-9a35-7663af661e15 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:59:51.533001 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cba15b-69cf-485e-9a35-7663af661e15 service placement] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:59:51.542642 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1498/2950] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:59:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:59:55.045018 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2460c4-7881-4b20-9b1e-bc7e273649b7 service neutron] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:59:55.058563 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2460c4-7881-4b20-9b1e-bc7e273649b7 service neutron] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:59:55.063885 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a571a83-765c-4d30-8c3a-fa6f32498757 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76ca877bee9041d28ad41e45ef3dd18c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:59:55.064539 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a571a83-765c-4d30-8c3a-fa6f32498757 service placement] REQUEST_METHOD: `GET` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:59:55.064825 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a571a83-765c-4d30-8c3a-fa6f32498757 service placement] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:59:55.065105 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a571a83-765c-4d30-8c3a-fa6f32498757 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:59:55.066872 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-9a571a83-765c-4d30-8c3a-fa6f32498757 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:59:55.078693 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a571a83-765c-4d30-8c3a-fa6f32498757 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:59:55.080316 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a571a83-765c-4d30-8c3a-fa6f32498757 service placement] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:59:55.090387 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1453/2951] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 17:59:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 17:59:55.148813 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cba15b-69cf-485e-9a35-7663af661e15 service placement] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:59:55.150881 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-92cba15b-69cf-485e-9a35-7663af661e15 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:59:55.168099 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cba15b-69cf-485e-9a35-7663af661e15 service placement] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:59:55.172891 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71cf513-220b-48f4-a851-9de7a75676d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:59:55.174330 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71cf513-220b-48f4-a851-9de7a75676d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:59:55.174612 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71cf513-220b-48f4-a851-9de7a75676d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:59:55.174879 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71cf513-220b-48f4-a851-9de7a75676d9 admin admin] PATH_INFO: `/v3/users/d2c4d7dd66614e52a006d1c6fb9d2fec` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:59:55.176102 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71cf513-220b-48f4-a851-9de7a75676d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d2c4d7dd66614e52a006d1c6fb9d2fec)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:59:55.178243 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71cf513-220b-48f4-a851-9de7a75676d9 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:59:55.343352 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-d71cf513-220b-48f4-a851-9de7a75676d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd2c4d7dd66614e52a006d1c6fb9d2fec'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:59:55.350973 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-d71cf513-220b-48f4-a851-9de7a75676d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd2c4d7dd66614e52a006d1c6fb9d2fec'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:59:55.351642 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-d71cf513-220b-48f4-a851-9de7a75676d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd2c4d7dd66614e52a006d1c6fb9d2fec'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:59:55.363767 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-d71cf513-220b-48f4-a851-9de7a75676d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd2c4d7dd66614e52a006d1c6fb9d2fec'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:59:55.445324 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-d71cf513-220b-48f4-a851-9de7a75676d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd2c4d7dd66614e52a006d1c6fb9d2fec'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:59:55.451467 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1499/2952] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:59:55 2022] DELETE /identity/v3/users/d2c4d7dd66614e52a006d1c6fb9d2fec => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:59:55.704440 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a571a83-765c-4d30-8c3a-fa6f32498757 service placement] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:59:55.715455 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a571a83-765c-4d30-8c3a-fa6f32498757 service placement] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:59:55.781685 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc64aea-acf5-4f28-9fae-bca3922af07a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:59:55.782539 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc64aea-acf5-4f28-9fae-bca3922af07a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:59:55.782828 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc64aea-acf5-4f28-9fae-bca3922af07a admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:59:55.783138 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc64aea-acf5-4f28-9fae-bca3922af07a admin admin] PATH_INFO: `/v3/projects/3d04d8f5cfdd466191e5c9e6fc5fc25e` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:59:55.784342 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc64aea-acf5-4f28-9fae-bca3922af07a admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d04d8f5cfdd466191e5c9e6fc5fc25e)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:59:55.786234 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc64aea-acf5-4f28-9fae-bca3922af07a admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:59:55.891122 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-fcc64aea-acf5-4f28-9fae-bca3922af07a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d04d8f5cfdd466191e5c9e6fc5fc25e', 'request_id': 'req-fcc64aea-acf5-4f28-9fae-bca3922af07a'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:59:55.898069 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-fcc64aea-acf5-4f28-9fae-bca3922af07a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d04d8f5cfdd466191e5c9e6fc5fc25e', 'request_id': 'req-fcc64aea-acf5-4f28-9fae-bca3922af07a'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:59:55.966158 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-fcc64aea-acf5-4f28-9fae-bca3922af07a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d04d8f5cfdd466191e5c9e6fc5fc25e', 'request_id': 'req-fcc64aea-acf5-4f28-9fae-bca3922af07a'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:59:55.978121 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-fcc64aea-acf5-4f28-9fae-bca3922af07a admin admin] The token cache is being invalidate because project 3d04d8f5cfdd466191e5c9e6fc5fc25e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:59:55.978417 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-fcc64aea-acf5-4f28-9fae-bca3922af07a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:59:55.979056 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-fcc64aea-acf5-4f28-9fae-bca3922af07a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:59:55.980696 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1454/2953] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:59:55 2022] DELETE /identity/v3/projects/3d04d8f5cfdd466191e5c9e6fc5fc25e => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:59:55.984942 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71cf513-220b-48f4-a851-9de7a75676d9 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:59:55.986286 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-d71cf513-220b-48f4-a851-9de7a75676d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:59:56.000342 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71cf513-220b-48f4-a851-9de7a75676d9 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:59:56.067301 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28a98c-a658-4ac3-863e-41b561a4a1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:59:56.068222 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28a98c-a658-4ac3-863e-41b561a4a1d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:59:56.068509 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28a98c-a658-4ac3-863e-41b561a4a1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:59:56.068795 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc28a98c-a658-4ac3-863e-41b561a4a1d4 admin admin] PATH_INFO: `/v3/users/f15f7f670eef426f8d12a0ea10c5b3de` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:59:56.069981 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc28a98c-a658-4ac3-863e-41b561a4a1d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f15f7f670eef426f8d12a0ea10c5b3de)` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:59:56.071910 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc28a98c-a658-4ac3-863e-41b561a4a1d4 admin admin] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:59:56.224710 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-dc28a98c-a658-4ac3-863e-41b561a4a1d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f15f7f670eef426f8d12a0ea10c5b3de'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:59:56.231592 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-dc28a98c-a658-4ac3-863e-41b561a4a1d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f15f7f670eef426f8d12a0ea10c5b3de'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:59:56.232281 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-dc28a98c-a658-4ac3-863e-41b561a4a1d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f15f7f670eef426f8d12a0ea10c5b3de'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:59:56.237283 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:59:56.245756 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-dc28a98c-a658-4ac3-863e-41b561a4a1d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f15f7f670eef426f8d12a0ea10c5b3de'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:59:56.325210 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.notifications [None req-dc28a98c-a658-4ac3-863e-41b561a4a1d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f15f7f670eef426f8d12a0ea10c5b3de'} {{(pid=80469) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:59:56.331344 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1500/2954] 10.222.0.40 () {58 vars in 1223 bytes} [Wed Jul 6 17:59:55 2022] DELETE /identity/v3/users/f15f7f670eef426f8d12a0ea10c5b3de => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 06 17:59:56.572553 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc64aea-acf5-4f28-9fae-bca3922af07a admin admin] Authenticating user token {{(pid=80470) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 17:59:56.573985 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.fernet_utils [None req-fcc64aea-acf5-4f28-9fae-bca3922af07a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80470) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 17:59:56.589965 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc64aea-acf5-4f28-9fae-bca3922af07a admin admin] Validating token access rules against request {{(pid=80470) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 17:59:56.604770 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 17:59:56.645971 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c371784e-46d4-47f3-8484-00ba1b918009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb49b9dd86cd415bb41412ca15c4d0ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '18f61d57ca574df5935e41a1653e029c', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80470) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 17:59:56.646833 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c371784e-46d4-47f3-8484-00ba1b918009 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 17:59:56.647114 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c371784e-46d4-47f3-8484-00ba1b918009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 17:59:56.647386 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c371784e-46d4-47f3-8484-00ba1b918009 admin admin] PATH_INFO: `/v3/projects/1f649aa2aae043788c8baf23c02d4414` {{(pid=80470) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 17:59:56.648598 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c371784e-46d4-47f3-8484-00ba1b918009 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f649aa2aae043788c8baf23c02d4414)` {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 17:59:56.650519 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c371784e-46d4-47f3-8484-00ba1b918009 admin admin] RBAC: Authorization granted {{(pid=80470) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 17:59:56.765043 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-c371784e-46d4-47f3-8484-00ba1b918009 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f649aa2aae043788c8baf23c02d4414', 'request_id': 'req-c371784e-46d4-47f3-8484-00ba1b918009'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:59:56.771613 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-c371784e-46d4-47f3-8484-00ba1b918009 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f649aa2aae043788c8baf23c02d4414', 'request_id': 'req-c371784e-46d4-47f3-8484-00ba1b918009'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:59:56.834273 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-c371784e-46d4-47f3-8484-00ba1b918009 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f649aa2aae043788c8baf23c02d4414', 'request_id': 'req-c371784e-46d4-47f3-8484-00ba1b918009'} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:59:56.845247 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-c371784e-46d4-47f3-8484-00ba1b918009 admin admin] The token cache is being invalidate because project 1f649aa2aae043788c8baf23c02d4414 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80470) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 06 17:59:56.846171 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-c371784e-46d4-47f3-8484-00ba1b918009 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:59:56.846806 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG keystone.notifications [None req-c371784e-46d4-47f3-8484-00ba1b918009 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80470) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 06 17:59:56.848520 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: [pid: 80470|app: 0|req: 1455/2955] 10.222.0.40 () {58 vars in 1232 bytes} [Wed Jul 6 17:59:56 2022] DELETE /identity/v3/projects/1f649aa2aae043788c8baf23c02d4414 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 06 18:00:06.451233 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:DELETE=3,keystone:SELECT=6,keystone:INSERT=1 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 18:00:06.842158 dv-cinder-806605-6-iscsi devstack@keystone.service[80470]: DEBUG dbcounter [-] [80470] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80470) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 06 18:03:05.118877 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28a98c-a658-4ac3-863e-41b561a4a1d4 admin admin] Authenticating user token {{(pid=80469) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 06 18:03:05.120357 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-dc28a98c-a658-4ac3-863e-41b561a4a1d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 18:03:05.199855 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc28a98c-a658-4ac3-863e-41b561a4a1d4 admin admin] Validating token access rules against request {{(pid=80469) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 06 18:03:05.258344 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c553e41-e5c8-41fc-b1f8-f4edce4bd818 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '76ca877bee9041d28ad41e45ef3dd18c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7938b0346d4240ab89774327d09a89ae', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80469) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 06 18:03:05.259002 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c553e41-e5c8-41fc-b1f8-f4edce4bd818 service placement] REQUEST_METHOD: `GET` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 06 18:03:05.259291 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c553e41-e5c8-41fc-b1f8-f4edce4bd818 service placement] SCRIPT_NAME: `/identity` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 06 18:03:05.259574 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c553e41-e5c8-41fc-b1f8-f4edce4bd818 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80469) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 06 18:03:05.261423 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.fernet_utils [None req-4c553e41-e5c8-41fc-b1f8-f4edce4bd818 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80469) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 06 18:03:05.273084 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c553e41-e5c8-41fc-b1f8-f4edce4bd818 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 06 18:03:05.274661 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c553e41-e5c8-41fc-b1f8-f4edce4bd818 service placement] RBAC: Authorization granted {{(pid=80469) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 06 18:03:05.284583 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: [pid: 80469|app: 0|req: 1501/2956] 10.222.0.40 () {60 vars in 1365 bytes} [Wed Jul 6 18:03:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 06 18:03:15.280293 dv-cinder-806605-6-iscsi devstack@keystone.service[80469]: DEBUG dbcounter [-] [80469] Writing DB stats keystone:SELECT=63 {{(pid=80469) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}